[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785992227050689-9601-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 04:57:07 dist-test-slave-x4qh krb5kdc[9608](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 04:57:07 dist-test-slave-x4qh krb5kdc[9608](info): set up 2 sockets
Aug 06 04:57:07 dist-test-slave-x4qh krb5kdc[9608](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260806 04:57:08.681385 9601 mini_kdc.cc:121] Time spent starting KDC: real 1.496s user 0.002s sys 0.016s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785992227050689-9601-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785992227050689-9601-0/krb5kdc/test-user.keytab.
Aug 06 04:57:08 dist-test-slave-x4qh krb5kdc[9608](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992228, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T04:57:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T04:57:08Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.96.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.96.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.96.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.126.keytab.
Entry for principal kudu/127.9.96.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.126.keytab.
I20260806 04:57:08.859607 9601 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
/tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785992227050689-9601-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785992227050689-9601-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785992227050689-9601-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785992227050689-9601-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.96.126:41435
--webserver_interface=127.9.96.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.126.keytab
--principal=kudu/127.9.96.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.96.84:33409
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.96.126:41435
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785992227050689-9601-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785992227050689-9601-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785992227050689-9601-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 04:57:09.226863 9627 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 04:57:09.227291 9627 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 04:57:09.227507 9627 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 04:57:09.234511 9627 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 04:57:09.234649 9627 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 04:57:09.234757 9627 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 04:57:09.234864 9627 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 04:57:09.234952 9627 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 04:57:09.244344 9627 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.96.84:33409
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785992227050689-9601-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785992227050689-9601-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.96.126:41435
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.126.keytab
--principal=kudu/127.9.96.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.96.126:41435
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785992227050689-9601-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.96.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785992227050689-9601-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 03:56:12 UTC on ccycloud.quasar-guijbm.root.comops.site
build id 4569618
ASAN enabled
I20260806 04:57:09.246212 9627 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 04:57:09.247970 9627 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 04:57:09.264537 9635 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:57:09.264528 9636 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:57:09.264994 9639 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:57:09.266233 9627 server_base.cc:1034] running on GCE node
I20260806 04:57:09.267223 9627 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 04:57:09.269234 9627 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 04:57:09.270467 9627 hybrid_clock.cc:648] HybridClock initialized: now 1785992229270406 us; error 89 us; skew 500 ppm
I20260806 04:57:09.270967 9627 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 04:57:09 dist-test-slave-x4qh krb5kdc[9608](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992229, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.96.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 04:57:09.274331 9627 init.cc:377] Logged in from keytab as kudu/127.9.96.126@KRBTEST.COM (short username kudu)
I20260806 04:57:09.276600 9627 webserver.cc:460] Webserver started at http://127.9.96.126:39799/ using document root <none> and password file <none>
I20260806 04:57:09.277595 9627 fs_manager.cc:359] Metadata directory not provided
I20260806 04:57:09.277750 9627 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 04:57:09.278184 9627 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 04:57:09.281808 9627 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785992227050689-9601-0/minicluster-data/master-0/data/instance:
uuid: "96d3ed335985468199d5a5739b9ac7f7"
format_stamp: "Formatted at 2026-08-06 04:57:09 on dist-test-slave-x4qh"
server_key: "f4fcb718ce0961587e4d51501483bc75"
server_key_iv: "b4add98a3f0c2609356ede420cdeb40f"
server_key_version: "encryptionkey@0"
I20260806 04:57:09.282598 9627 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785992227050689-9601-0/minicluster-data/master-0/wal/instance:
uuid: "96d3ed335985468199d5a5739b9ac7f7"
format_stamp: "Formatted at 2026-08-06 04:57:09 on dist-test-slave-x4qh"
server_key: "f4fcb718ce0961587e4d51501483bc75"
server_key_iv: "b4add98a3f0c2609356ede420cdeb40f"
server_key_version: "encryptionkey@0"
I20260806 04:57:09.289615 9627 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.003s
I20260806 04:57:09.295277 9648 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 04:57:09.297678 9627 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.002s sys 0.001s
I20260806 04:57:09.297935 9627 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785992227050689-9601-0/minicluster-data/master-0/data,/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785992227050689-9601-0/minicluster-data/master-0/wal
uuid: "96d3ed335985468199d5a5739b9ac7f7"
format_stamp: "Formatted at 2026-08-06 04:57:09 on dist-test-slave-x4qh"
server_key: "f4fcb718ce0961587e4d51501483bc75"
server_key_iv: "b4add98a3f0c2609356ede420cdeb40f"
server_key_version: "encryptionkey@0"
I20260806 04:57:09.298142 9627 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785992227050689-9601-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785992227050689-9601-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785992227050689-9601-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 04:57:09.347213 9627 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 04:57:09.351919 9627 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 04:57:09.352280 9627 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 04:57:09.373831 9627 rpc_server.cc:307] RPC server started. Bound to: 127.9.96.126:41435
I20260806 04:57:09.373905 9722 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.96.126:41435 every 8 connection(s)
I20260806 04:57:09.375976 9627 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785992227050689-9601-0/minicluster-data/master-0/data/info.pb
I20260806 04:57:09.378227 9601 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu as pid 9627
I20260806 04:57:09.378628 9601 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785992227050689-9601-0/minicluster-data/master-0/wal/instance
I20260806 04:57:09.382184 9601 external_mini_cluster.cc:1428] Setting key ded69d32e4234b7254677b7a3ea9965f
I20260806 04:57:09.384491 9723 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 04:57:09.406723 9723 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 96d3ed335985468199d5a5739b9ac7f7: Bootstrap starting.
I20260806 04:57:09.413414 9723 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 96d3ed335985468199d5a5739b9ac7f7: Neither blocks nor log segments found. Creating new log.
I20260806 04:57:09.415407 9723 log.cc:824] T 00000000000000000000000000000000 P 96d3ed335985468199d5a5739b9ac7f7: Log is configured to *not* fsync() on all Append() calls
I20260806 04:57:09.420169 9723 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 96d3ed335985468199d5a5739b9ac7f7: No bootstrap required, opened a new log
Aug 06 04:57:09 dist-test-slave-x4qh krb5kdc[9608](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992228, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.96.126@KRBTEST.COM
I20260806 04:57:09.432647 9723 raft_consensus.cc:348] T 00000000000000000000000000000000 P 96d3ed335985468199d5a5739b9ac7f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96d3ed335985468199d5a5739b9ac7f7" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 41435 } }
I20260806 04:57:09.433216 9723 raft_consensus.cc:374] T 00000000000000000000000000000000 P 96d3ed335985468199d5a5739b9ac7f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 04:57:09.433418 9723 raft_consensus.cc:729] T 00000000000000000000000000000000 P 96d3ed335985468199d5a5739b9ac7f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 96d3ed335985468199d5a5739b9ac7f7, State: Initialized, Role: FOLLOWER
I20260806 04:57:09.434861 9723 consensus_queue.cc:260] T 00000000000000000000000000000000 P 96d3ed335985468199d5a5739b9ac7f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96d3ed335985468199d5a5739b9ac7f7" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 41435 } }
I20260806 04:57:09.435104 9726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:57:09.392357 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51086 (local address 127.9.96.126:41435)
0806 04:57:09.393160 (+ 803us) server_negotiation.cc:206] Beginning negotiation
0806 04:57:09.393169 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 04:57:09.393209 (+ 40us) server_negotiation.cc:407] Connection header received
0806 04:57:09.404927 (+ 11718us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:57:09.404946 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:57:09.405401 (+ 455us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:57:09.405737 (+ 336us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:57:09.410936 (+ 5199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:57:09.412411 (+ 1475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:57:09.415357 (+ 2946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:57:09.416373 (+ 1016us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:57:09.416662 (+ 289us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:57:09.424115 (+ 7453us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:57:09.424199 (+ 84us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:57:09.424245 (+ 46us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:57:09.424289 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:57:09.428596 (+ 4307us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:57:09.429453 (+ 857us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:57:09.429463 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:57:09.429474 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:57:09.429772 (+ 298us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:57:09.430218 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:57:09.430229 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:57:09.430233 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:57:09.431140 (+ 907us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:57:09.431589 (+ 449us) server_negotiation.cc:1035] Waiting for connection context
0806 04:57:09.432460 (+ 871us) server_negotiation.cc:299] Negotiation successful
0806 04:57:09.432950 (+ 490us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":577,"thread_start_us":372,"threads_started":1}
I20260806 04:57:09.435282 9723 raft_consensus.cc:388] T 00000000000000000000000000000000 P 96d3ed335985468199d5a5739b9ac7f7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 04:57:09.435611 9723 raft_consensus.cc:482] T 00000000000000000000000000000000 P 96d3ed335985468199d5a5739b9ac7f7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 04:57:09.435806 9723 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 96d3ed335985468199d5a5739b9ac7f7 [term 0 FOLLOWER]: Advancing to term 1
I20260806 04:57:09.438302 9723 raft_consensus.cc:504] T 00000000000000000000000000000000 P 96d3ed335985468199d5a5739b9ac7f7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96d3ed335985468199d5a5739b9ac7f7" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 41435 } }
I20260806 04:57:09.439198 9723 leader_election.cc:302] T 00000000000000000000000000000000 P 96d3ed335985468199d5a5739b9ac7f7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 96d3ed335985468199d5a5739b9ac7f7; no voters:
I20260806 04:57:09.440131 9723 leader_election.cc:288] T 00000000000000000000000000000000 P 96d3ed335985468199d5a5739b9ac7f7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 04:57:09.440469 9730 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 96d3ed335985468199d5a5739b9ac7f7 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 04:57:09.441254 9730 raft_consensus.cc:686] T 00000000000000000000000000000000 P 96d3ed335985468199d5a5739b9ac7f7 [term 1 LEADER]: Becoming Leader. State: Replica: 96d3ed335985468199d5a5739b9ac7f7, State: Running, Role: LEADER
I20260806 04:57:09.441993 9730 consensus_queue.cc:237] T 00000000000000000000000000000000 P 96d3ed335985468199d5a5739b9ac7f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96d3ed335985468199d5a5739b9ac7f7" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 41435 } }
I20260806 04:57:09.442906 9723 sys_catalog.cc:564] T 00000000000000000000000000000000 P 96d3ed335985468199d5a5739b9ac7f7 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 04:57:09.447469 9732 sys_catalog.cc:455] T 00000000000000000000000000000000 P 96d3ed335985468199d5a5739b9ac7f7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 96d3ed335985468199d5a5739b9ac7f7. Latest consensus state: current_term: 1 leader_uuid: "96d3ed335985468199d5a5739b9ac7f7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96d3ed335985468199d5a5739b9ac7f7" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 41435 } } }
I20260806 04:57:09.447660 9731 sys_catalog.cc:455] T 00000000000000000000000000000000 P 96d3ed335985468199d5a5739b9ac7f7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "96d3ed335985468199d5a5739b9ac7f7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96d3ed335985468199d5a5739b9ac7f7" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 41435 } } }
I20260806 04:57:09.448038 9732 sys_catalog.cc:458] T 00000000000000000000000000000000 P 96d3ed335985468199d5a5739b9ac7f7 [sys.catalog]: This master's current role is: LEADER
I20260806 04:57:09.448098 9731 sys_catalog.cc:458] T 00000000000000000000000000000000 P 96d3ed335985468199d5a5739b9ac7f7 [sys.catalog]: This master's current role is: LEADER
I20260806 04:57:09.451391 9736 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 04:57:09.459635 9736 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 04:57:09.473886 9736 catalog_manager.cc:1347] Generated new cluster ID: e736498d2e63478e9d9a3a1a6fa85260
I20260806 04:57:09.474068 9736 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 04:57:09.484481 9736 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 04:57:09.486387 9736 catalog_manager.cc:1504] Loading token signing keys...
I20260806 04:57:09.496634 9736 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 96d3ed335985468199d5a5739b9ac7f7: Generated new TSK 0
I20260806 04:57:09.497607 9736 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.9.96.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.96.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.96.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.65.keytab.
Entry for principal kudu/127.9.96.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.65.keytab.
I20260806 04:57:09.546650 9601 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
/tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785992227050689-9601-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785992227050689-9601-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785992227050689-9601-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785992227050689-9601-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.96.65:0
--local_ip_for_outbound_sockets=127.9.96.65
--webserver_interface=127.9.96.65
--webserver_port=0
--tserver_master_addrs=127.9.96.126:41435
--keytab_file=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.65.keytab
--principal=kudu/127.9.96.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.96.84:33409
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785992227050689-9601-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785992227050689-9601-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785992227050689-9601-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 04:57:09.896682 9757 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 04:57:09.896932 9757 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 04:57:09.897050 9757 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 04:57:09.904598 9757 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 04:57:09.904731 9757 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 04:57:09.904929 9757 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.96.65
I20260806 04:57:09.914425 9757 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.96.84:33409
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785992227050689-9601-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785992227050689-9601-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.65.keytab
--principal=kudu/127.9.96.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.96.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785992227050689-9601-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.96.65
--webserver_port=0
--tserver_master_addrs=127.9.96.126:41435
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.96.65
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785992227050689-9601-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 03:56:12 UTC on ccycloud.quasar-guijbm.root.comops.site
build id 4569618
ASAN enabled
I20260806 04:57:09.915669 9757 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 04:57:09.916981 9757 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 04:57:09.930895 9768 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:57:09.930872 9763 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:57:09.931620 9762 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:57:09.931743 9757 server_base.cc:1034] running on GCE node
I20260806 04:57:09.932353 9757 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 04:57:09.933188 9757 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 04:57:09.934392 9757 hybrid_clock.cc:648] HybridClock initialized: now 1785992229934338 us; error 52 us; skew 500 ppm
I20260806 04:57:09.934700 9757 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 04:57:09 dist-test-slave-x4qh krb5kdc[9608](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992229, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.96.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 04:57:09.938079 9757 init.cc:377] Logged in from keytab as kudu/127.9.96.65@KRBTEST.COM (short username kudu)
I20260806 04:57:09.939527 9757 webserver.cc:460] Webserver started at http://127.9.96.65:35857/ using document root <none> and password file <none>
I20260806 04:57:09.940218 9757 fs_manager.cc:359] Metadata directory not provided
I20260806 04:57:09.940387 9757 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 04:57:09.940706 9757 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 04:57:09.943817 9757 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785992227050689-9601-0/minicluster-data/ts-0/data/instance:
uuid: "cb62e221b84b4a4ba238137a859aa71f"
format_stamp: "Formatted at 2026-08-06 04:57:09 on dist-test-slave-x4qh"
server_key: "37916cf79fb2b1bac644ebb1a6c267b7"
server_key_iv: "aaadc4302aa8419465b99150df904e56"
server_key_version: "encryptionkey@0"
I20260806 04:57:09.944607 9757 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785992227050689-9601-0/minicluster-data/ts-0/wal/instance:
uuid: "cb62e221b84b4a4ba238137a859aa71f"
format_stamp: "Formatted at 2026-08-06 04:57:09 on dist-test-slave-x4qh"
server_key: "37916cf79fb2b1bac644ebb1a6c267b7"
server_key_iv: "aaadc4302aa8419465b99150df904e56"
server_key_version: "encryptionkey@0"
I20260806 04:57:09.949663 9757 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.004s
I20260806 04:57:09.953202 9779 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 04:57:09.954447 9757 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260806 04:57:09.954620 9757 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785992227050689-9601-0/minicluster-data/ts-0/data,/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785992227050689-9601-0/minicluster-data/ts-0/wal
uuid: "cb62e221b84b4a4ba238137a859aa71f"
format_stamp: "Formatted at 2026-08-06 04:57:09 on dist-test-slave-x4qh"
server_key: "37916cf79fb2b1bac644ebb1a6c267b7"
server_key_iv: "aaadc4302aa8419465b99150df904e56"
server_key_version: "encryptionkey@0"
I20260806 04:57:09.954824 9757 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785992227050689-9601-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785992227050689-9601-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785992227050689-9601-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 04:57:09.985803 9757 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 04:57:09.989639 9757 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 04:57:09.989945 9757 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 04:57:09.991050 9757 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 04:57:09.992643 9757 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 04:57:09.992755 9757 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 04:57:09.992970 9757 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 04:57:09.993065 9757 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 04:57:10.029399 9757 rpc_server.cc:307] RPC server started. Bound to: 127.9.96.65:33055
I20260806 04:57:10.029449 9961 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.96.65:33055 every 8 connection(s)
I20260806 04:57:10.030768 9757 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785992227050689-9601-0/minicluster-data/ts-0/data/info.pb
I20260806 04:57:10.034507 9601 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu as pid 9757
I20260806 04:57:10.034735 9601 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785992227050689-9601-0/minicluster-data/ts-0/wal/instance
I20260806 04:57:10.035418 9601 external_mini_cluster.cc:1428] Setting key 1dbb46ddb5989b90ec6ec19b8ce84d9d
Aug 06 04:57:10 dist-test-slave-x4qh krb5kdc[9608](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992229, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.96.65@KRBTEST.COM for kudu/127.9.96.126@KRBTEST.COM
I20260806 04:57:10.052781 9966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:57:10.033951 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.96.65:51311 (local address 127.9.96.126:41435)
0806 04:57:10.034533 (+ 582us) server_negotiation.cc:206] Beginning negotiation
0806 04:57:10.034540 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 04:57:10.035555 (+ 1015us) server_negotiation.cc:407] Connection header received
0806 04:57:10.037448 (+ 1893us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:57:10.037454 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:57:10.037531 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:57:10.037646 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:57:10.039019 (+ 1373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:57:10.040238 (+ 1219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:57:10.041774 (+ 1536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:57:10.042496 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:57:10.042646 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:57:10.046572 (+ 3926us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:57:10.046583 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:57:10.046586 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:57:10.046644 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:57:10.049334 (+ 2690us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:57:10.050115 (+ 781us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:57:10.050124 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:57:10.050131 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:57:10.050329 (+ 198us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:57:10.050761 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:57:10.050769 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:57:10.050775 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:57:10.051188 (+ 413us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:57:10.051360 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0806 04:57:10.052065 (+ 705us) server_negotiation.cc:299] Negotiation successful
0806 04:57:10.052333 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":446,"thread_start_us":287,"threads_started":1}
I20260806 04:57:10.053480 9967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:57:10.034487 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:41435 (local address 127.9.96.65:51311)
0806 04:57:10.035392 (+ 905us) negotiation.cc:107] Waiting for socket to connect
0806 04:57:10.035448 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0806 04:57:10.037198 (+ 1750us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:57:10.037800 (+ 602us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:57:10.037808 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:57:10.038301 (+ 493us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:57:10.038832 (+ 531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:57:10.038848 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:57:10.040412 (+ 1564us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:57:10.040416 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:57:10.041630 (+ 1214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:57:10.041641 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:57:10.042724 (+ 1083us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:57:10.042729 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:57:10.043014 (+ 285us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:57:10.043752 (+ 738us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:57:10.043773 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:57:10.046337 (+ 2564us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 04:57:10.049487 (+ 3150us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:57:10.049497 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:57:10.049512 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:57:10.049930 (+ 418us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:57:10.050470 (+ 540us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:57:10.050474 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:57:10.050476 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:57:10.050650 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:57:10.051311 (+ 661us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 04:57:10.051316 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 04:57:10.051704 (+ 388us) client_negotiation.cc:769] Sending connection context
0806 04:57:10.051936 (+ 232us) client_negotiation.cc:240] Negotiation successful
0806 04:57:10.052287 (+ 351us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":599,"thread_start_us":399,"threads_started":1}
WARNING: no policy specified for kudu/127.9.96.66@KRBTEST.COM; defaulting to no policy
I20260806 04:57:10.055197 9963 heartbeater.cc:347] Connected to a master server at 127.9.96.126:41435
I20260806 04:57:10.055778 9963 heartbeater.cc:464] Registering TS with master...
I20260806 04:57:10.056541 9963 heartbeater.cc:511] Master 127.9.96.126:41435 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.96.66@KRBTEST.COM" created.
I20260806 04:57:10.061038 9676 ts_manager.cc:194] Registered new tserver with Master: cb62e221b84b4a4ba238137a859aa71f (127.9.96.65:33055)
I20260806 04:57:10.063827 9676 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.96.65@KRBTEST.COM'} at 127.9.96.65:51311
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.96.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.66.keytab.
Entry for principal kudu/127.9.96.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.66.keytab.
I20260806 04:57:10.082432 9601 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
/tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785992227050689-9601-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785992227050689-9601-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785992227050689-9601-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785992227050689-9601-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.96.66:0
--local_ip_for_outbound_sockets=127.9.96.66
--webserver_interface=127.9.96.66
--webserver_port=0
--tserver_master_addrs=127.9.96.126:41435
--keytab_file=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.66.keytab
--principal=kudu/127.9.96.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.96.84:33409
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785992227050689-9601-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785992227050689-9601-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785992227050689-9601-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 04:57:10.414744 9970 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 04:57:10.415014 9970 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 04:57:10.415177 9970 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 04:57:10.422430 9970 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 04:57:10.422550 9970 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 04:57:10.422773 9970 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.96.66
I20260806 04:57:10.432416 9970 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.96.84:33409
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785992227050689-9601-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785992227050689-9601-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.66.keytab
--principal=kudu/127.9.96.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.96.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785992227050689-9601-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.96.66
--webserver_port=0
--tserver_master_addrs=127.9.96.126:41435
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.96.66
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785992227050689-9601-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 03:56:12 UTC on ccycloud.quasar-guijbm.root.comops.site
build id 4569618
ASAN enabled
I20260806 04:57:10.433630 9970 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 04:57:10.435000 9970 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 04:57:10.448076 9977 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:57:10.448048 9978 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:57:10.448351 9985 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:57:10.449151 9970 server_base.cc:1034] running on GCE node
I20260806 04:57:10.449741 9970 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 04:57:10.450613 9970 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 04:57:10.451812 9970 hybrid_clock.cc:648] HybridClock initialized: now 1785992230451753 us; error 44 us; skew 500 ppm
I20260806 04:57:10.452137 9970 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 04:57:10 dist-test-slave-x4qh krb5kdc[9608](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992230, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.96.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 04:57:10.458027 9970 init.cc:377] Logged in from keytab as kudu/127.9.96.66@KRBTEST.COM (short username kudu)
I20260806 04:57:10.459640 9970 webserver.cc:460] Webserver started at http://127.9.96.66:37053/ using document root <none> and password file <none>
I20260806 04:57:10.460275 9970 fs_manager.cc:359] Metadata directory not provided
I20260806 04:57:10.460379 9970 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 04:57:10.460651 9970 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 04:57:10.463800 9970 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785992227050689-9601-0/minicluster-data/ts-1/data/instance:
uuid: "9a4716b20ac848c0a0f6be0f35c4c294"
format_stamp: "Formatted at 2026-08-06 04:57:10 on dist-test-slave-x4qh"
server_key: "465a73f259cd2d3cd7bbd52225c5cdfb"
server_key_iv: "660b8d38a4dded62a16c8abb14c9c712"
server_key_version: "encryptionkey@0"
I20260806 04:57:10.464550 9970 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785992227050689-9601-0/minicluster-data/ts-1/wal/instance:
uuid: "9a4716b20ac848c0a0f6be0f35c4c294"
format_stamp: "Formatted at 2026-08-06 04:57:10 on dist-test-slave-x4qh"
server_key: "465a73f259cd2d3cd7bbd52225c5cdfb"
server_key_iv: "660b8d38a4dded62a16c8abb14c9c712"
server_key_version: "encryptionkey@0"
I20260806 04:57:10.469640 9970 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.005s
I20260806 04:57:10.473034 9996 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 04:57:10.474332 9970 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260806 04:57:10.474509 9970 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785992227050689-9601-0/minicluster-data/ts-1/data,/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785992227050689-9601-0/minicluster-data/ts-1/wal
uuid: "9a4716b20ac848c0a0f6be0f35c4c294"
format_stamp: "Formatted at 2026-08-06 04:57:10 on dist-test-slave-x4qh"
server_key: "465a73f259cd2d3cd7bbd52225c5cdfb"
server_key_iv: "660b8d38a4dded62a16c8abb14c9c712"
server_key_version: "encryptionkey@0"
I20260806 04:57:10.474682 9970 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785992227050689-9601-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785992227050689-9601-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785992227050689-9601-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 04:57:10.487269 9970 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 04:57:10.491079 9970 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 04:57:10.491353 9970 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 04:57:10.492303 9970 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 04:57:10.493580 9970 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 04:57:10.493656 9970 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 04:57:10.493824 9970 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 04:57:10.493889 9970 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 04:57:10.528937 9970 rpc_server.cc:307] RPC server started. Bound to: 127.9.96.66:36881
I20260806 04:57:10.529038 10160 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.96.66:36881 every 8 connection(s)
I20260806 04:57:10.530228 9970 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785992227050689-9601-0/minicluster-data/ts-1/data/info.pb
I20260806 04:57:10.537724 9601 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu as pid 9970
I20260806 04:57:10.537935 9601 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785992227050689-9601-0/minicluster-data/ts-1/wal/instance
I20260806 04:57:10.538395 9601 external_mini_cluster.cc:1428] Setting key 6c7059d873e70716fd91ff080fefe7d1
Aug 06 04:57:10 dist-test-slave-x4qh krb5kdc[9608](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992230, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.96.66@KRBTEST.COM for kudu/127.9.96.126@KRBTEST.COM
I20260806 04:57:10.551437 9966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:57:10.532841 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.96.66:33983 (local address 127.9.96.126:41435)
0806 04:57:10.533017 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0806 04:57:10.533021 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 04:57:10.534141 (+ 1120us) server_negotiation.cc:407] Connection header received
0806 04:57:10.536036 (+ 1895us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:57:10.536041 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:57:10.536132 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:57:10.536265 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:57:10.537705 (+ 1440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:57:10.538646 (+ 941us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:57:10.540831 (+ 2185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:57:10.541277 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:57:10.541409 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:57:10.545143 (+ 3734us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:57:10.545152 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:57:10.545160 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:57:10.545205 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:57:10.548128 (+ 2923us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:57:10.549036 (+ 908us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:57:10.549040 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:57:10.549042 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:57:10.549181 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:57:10.549665 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:57:10.549668 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:57:10.549670 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:57:10.549931 (+ 261us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:57:10.550042 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0806 04:57:10.550896 (+ 854us) server_negotiation.cc:299] Negotiation successful
0806 04:57:10.551067 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260806 04:57:10.552784 10166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:57:10.533255 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:41435 (local address 127.9.96.66:33983)
0806 04:57:10.533960 (+ 705us) negotiation.cc:107] Waiting for socket to connect
0806 04:57:10.534010 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0806 04:57:10.535778 (+ 1768us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:57:10.536488 (+ 710us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:57:10.536496 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:57:10.536973 (+ 477us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:57:10.537526 (+ 553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:57:10.537544 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:57:10.538812 (+ 1268us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:57:10.538816 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:57:10.540689 (+ 1873us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:57:10.540702 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:57:10.541510 (+ 808us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:57:10.541517 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:57:10.541735 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:57:10.542465 (+ 730us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:57:10.542488 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:57:10.544920 (+ 2432us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 04:57:10.548270 (+ 3350us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:57:10.548281 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:57:10.548297 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:57:10.548919 (+ 622us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:57:10.549288 (+ 369us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:57:10.549296 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:57:10.549303 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:57:10.549551 (+ 248us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:57:10.550069 (+ 518us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 04:57:10.550079 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 04:57:10.550502 (+ 423us) client_negotiation.cc:769] Sending connection context
0806 04:57:10.550804 (+ 302us) client_negotiation.cc:240] Negotiation successful
0806 04:57:10.551359 (+ 555us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":488,"thread_start_us":282,"threads_started":1}
I20260806 04:57:10.554246 10162 heartbeater.cc:347] Connected to a master server at 127.9.96.126:41435
I20260806 04:57:10.554571 10162 heartbeater.cc:464] Registering TS with master...
I20260806 04:57:10.555274 10162 heartbeater.cc:511] Master 127.9.96.126:41435 requested a full tablet report, sending...
I20260806 04:57:10.556746 9676 ts_manager.cc:194] Registered new tserver with Master: 9a4716b20ac848c0a0f6be0f35c4c294 (127.9.96.66:36881)
WARNING: no policy specified for kudu/127.9.96.67@KRBTEST.COM; defaulting to no policy
I20260806 04:57:10.557585 9676 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.96.66@KRBTEST.COM'} at 127.9.96.66:33983
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.96.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.96.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.67.keytab.
Entry for principal kudu/127.9.96.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.67.keytab.
I20260806 04:57:10.582954 9601 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
/tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785992227050689-9601-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785992227050689-9601-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785992227050689-9601-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785992227050689-9601-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.96.67:0
--local_ip_for_outbound_sockets=127.9.96.67
--webserver_interface=127.9.96.67
--webserver_port=0
--tserver_master_addrs=127.9.96.126:41435
--keytab_file=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.67.keytab
--principal=kudu/127.9.96.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.96.84:33409
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785992227050689-9601-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785992227050689-9601-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785992227050689-9601-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 04:57:10.909265 10171 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 04:57:10.909534 10171 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 04:57:10.909734 10171 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 04:57:10.917047 10171 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 04:57:10.917160 10171 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 04:57:10.917356 10171 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.96.67
I20260806 04:57:10.927484 10171 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.96.84:33409
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785992227050689-9601-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785992227050689-9601-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.67.keytab
--principal=kudu/127.9.96.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.96.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785992227050689-9601-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.96.67
--webserver_port=0
--tserver_master_addrs=127.9.96.126:41435
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.96.67
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785992227050689-9601-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 03:56:12 UTC on ccycloud.quasar-guijbm.root.comops.site
build id 4569618
ASAN enabled
I20260806 04:57:10.928771 10171 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 04:57:10.930095 10171 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 04:57:10.943207 10178 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:57:10.943481 10179 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:57:10.943617 10181 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:57:10.944540 10171 server_base.cc:1034] running on GCE node
I20260806 04:57:10.945129 10171 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 04:57:10.946031 10171 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 04:57:10.947284 10171 hybrid_clock.cc:648] HybridClock initialized: now 1785992230947236 us; error 53 us; skew 500 ppm
I20260806 04:57:10.947674 10171 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 04:57:10 dist-test-slave-x4qh krb5kdc[9608](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992230, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.96.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 04:57:10.951117 10171 init.cc:377] Logged in from keytab as kudu/127.9.96.67@KRBTEST.COM (short username kudu)
I20260806 04:57:10.952700 10171 webserver.cc:460] Webserver started at http://127.9.96.67:43817/ using document root <none> and password file <none>
I20260806 04:57:10.953368 10171 fs_manager.cc:359] Metadata directory not provided
I20260806 04:57:10.953539 10171 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 04:57:10.953864 10171 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 04:57:10.956847 10171 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785992227050689-9601-0/minicluster-data/ts-2/data/instance:
uuid: "1531a23d7c834f089df1b3eab0fd686d"
format_stamp: "Formatted at 2026-08-06 04:57:10 on dist-test-slave-x4qh"
server_key: "1f2af023017a722a4c01402a261f8fe7"
server_key_iv: "d9d7002168be78bb2aa4b3e751aff68e"
server_key_version: "encryptionkey@0"
I20260806 04:57:10.957612 10171 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785992227050689-9601-0/minicluster-data/ts-2/wal/instance:
uuid: "1531a23d7c834f089df1b3eab0fd686d"
format_stamp: "Formatted at 2026-08-06 04:57:10 on dist-test-slave-x4qh"
server_key: "1f2af023017a722a4c01402a261f8fe7"
server_key_iv: "d9d7002168be78bb2aa4b3e751aff68e"
server_key_version: "encryptionkey@0"
I20260806 04:57:10.962675 10171 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260806 04:57:10.966045 10195 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 04:57:10.967257 10171 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 04:57:10.967453 10171 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785992227050689-9601-0/minicluster-data/ts-2/data,/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785992227050689-9601-0/minicluster-data/ts-2/wal
uuid: "1531a23d7c834f089df1b3eab0fd686d"
format_stamp: "Formatted at 2026-08-06 04:57:10 on dist-test-slave-x4qh"
server_key: "1f2af023017a722a4c01402a261f8fe7"
server_key_iv: "d9d7002168be78bb2aa4b3e751aff68e"
server_key_version: "encryptionkey@0"
I20260806 04:57:10.967615 10171 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785992227050689-9601-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785992227050689-9601-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785992227050689-9601-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 04:57:10.979990 10171 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 04:57:10.984236 10171 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 04:57:10.984491 10171 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 04:57:10.985337 10171 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 04:57:10.986523 10171 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 04:57:10.986596 10171 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 04:57:10.986680 10171 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 04:57:10.986725 10171 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 04:57:11.021307 10171 rpc_server.cc:307] RPC server started. Bound to: 127.9.96.67:37017
I20260806 04:57:11.021358 10366 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.96.67:37017 every 8 connection(s)
I20260806 04:57:11.022568 10171 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785992227050689-9601-0/minicluster-data/ts-2/data/info.pb
I20260806 04:57:11.029959 9601 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu as pid 10171
I20260806 04:57:11.030179 9601 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785992227050689-9601-0/minicluster-data/ts-2/wal/instance
I20260806 04:57:11.030632 9601 external_mini_cluster.cc:1428] Setting key 3500da092b505800662b6a000c35a5cd
Aug 06 04:57:11 dist-test-slave-x4qh krb5kdc[9608](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992230, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.96.67@KRBTEST.COM for kudu/127.9.96.126@KRBTEST.COM
I20260806 04:57:11.043464 9966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:57:11.025125 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.96.67:51111 (local address 127.9.96.126:41435)
0806 04:57:11.025302 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0806 04:57:11.025307 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 04:57:11.026423 (+ 1116us) server_negotiation.cc:407] Connection header received
0806 04:57:11.028274 (+ 1851us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:57:11.028279 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:57:11.028409 (+ 130us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:57:11.028528 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:57:11.030283 (+ 1755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:57:11.031466 (+ 1183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:57:11.033635 (+ 2169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:57:11.034357 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:57:11.034510 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:57:11.038112 (+ 3602us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:57:11.038117 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:57:11.038120 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:57:11.038152 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:57:11.040858 (+ 2706us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:57:11.041471 (+ 613us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:57:11.041475 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:57:11.041477 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:57:11.041599 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:57:11.041952 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:57:11.041956 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:57:11.041958 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:57:11.042205 (+ 247us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:57:11.042316 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0806 04:57:11.043016 (+ 700us) server_negotiation.cc:299] Negotiation successful
0806 04:57:11.043171 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260806 04:57:11.044368 10371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:57:11.025533 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:41435 (local address 127.9.96.67:51111)
0806 04:57:11.026244 (+ 711us) negotiation.cc:107] Waiting for socket to connect
0806 04:57:11.026281 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0806 04:57:11.028001 (+ 1720us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:57:11.028777 (+ 776us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:57:11.028785 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:57:11.029280 (+ 495us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:57:11.030015 (+ 735us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:57:11.030043 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:57:11.031656 (+ 1613us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:57:11.031662 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:57:11.033407 (+ 1745us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:57:11.033420 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:57:11.034702 (+ 1282us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:57:11.034713 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:57:11.035044 (+ 331us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:57:11.035732 (+ 688us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:57:11.035753 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:57:11.037912 (+ 2159us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 04:57:11.040970 (+ 3058us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:57:11.040976 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:57:11.040985 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:57:11.041374 (+ 389us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:57:11.041691 (+ 317us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:57:11.041694 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:57:11.041696 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:57:11.041859 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:57:11.042312 (+ 453us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 04:57:11.042317 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 04:57:11.042588 (+ 271us) client_negotiation.cc:769] Sending connection context
0806 04:57:11.042889 (+ 301us) client_negotiation.cc:240] Negotiation successful
0806 04:57:11.043230 (+ 341us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":492,"thread_start_us":287,"threads_started":1}
I20260806 04:57:11.045794 10367 heartbeater.cc:347] Connected to a master server at 127.9.96.126:41435
I20260806 04:57:11.046131 10367 heartbeater.cc:464] Registering TS with master...
I20260806 04:57:11.046975 10367 heartbeater.cc:511] Master 127.9.96.126:41435 requested a full tablet report, sending...
I20260806 04:57:11.048396 9676 ts_manager.cc:194] Registered new tserver with Master: 1531a23d7c834f089df1b3eab0fd686d (127.9.96.67:37017)
I20260806 04:57:11.049189 9676 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.96.67@KRBTEST.COM'} at 127.9.96.67:51111
I20260806 04:57:11.059521 9601 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 04:57:11.067026 9963 heartbeater.cc:503] Master 127.9.96.126:41435 was elected leader, sending a full tablet report...
Aug 06 04:57:11 dist-test-slave-x4qh krb5kdc[9608](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992231, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 06 04:57:11 dist-test-slave-x4qh krb5kdc[9608](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992231, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.9.96.65@KRBTEST.COM
I20260806 04:57:11.106408 10377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:57:11.091627 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37726 (local address 127.9.96.65:33055)
0806 04:57:11.092345 (+ 718us) server_negotiation.cc:206] Beginning negotiation
0806 04:57:11.092352 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 04:57:11.092416 (+ 64us) server_negotiation.cc:407] Connection header received
0806 04:57:11.092482 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:57:11.092488 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:57:11.092704 (+ 216us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:57:11.092833 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:57:11.094223 (+ 1390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:57:11.095206 (+ 983us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:57:11.096523 (+ 1317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:57:11.097071 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:57:11.097167 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:57:11.100998 (+ 3831us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:57:11.101009 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:57:11.101022 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:57:11.101055 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:57:11.103977 (+ 2922us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:57:11.104582 (+ 605us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:57:11.104588 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:57:11.104591 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:57:11.104708 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:57:11.105054 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:57:11.105057 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:57:11.105059 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:57:11.105379 (+ 320us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:57:11.105608 (+ 229us) server_negotiation.cc:1035] Waiting for connection context
0806 04:57:11.105820 (+ 212us) server_negotiation.cc:299] Negotiation successful
0806 04:57:11.106056 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":577,"thread_start_us":412,"threads_started":1}
W20260806 04:57:11.107869 9821 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:37726
Aug 06 04:57:11 dist-test-slave-x4qh krb5kdc[9608](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992231, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 06 04:57:11 dist-test-slave-x4qh krb5kdc[9608](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992231, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.96.65@KRBTEST.COM
I20260806 04:57:11.152976 10377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:57:11.138765 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37740 (local address 127.9.96.65:33055)
0806 04:57:11.138992 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0806 04:57:11.139000 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 04:57:11.139229 (+ 229us) server_negotiation.cc:407] Connection header received
0806 04:57:11.139470 (+ 241us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:57:11.139475 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:57:11.139569 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:57:11.139681 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:57:11.141477 (+ 1796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:57:11.142252 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:57:11.143598 (+ 1346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:57:11.144102 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:57:11.144200 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:57:11.147972 (+ 3772us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:57:11.147977 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:57:11.147980 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:57:11.148013 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:57:11.150702 (+ 2689us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:57:11.151259 (+ 557us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:57:11.151263 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:57:11.151265 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:57:11.151444 (+ 179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:57:11.151799 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:57:11.151806 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:57:11.151809 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:57:11.152058 (+ 249us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:57:11.152238 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0806 04:57:11.152474 (+ 236us) server_negotiation.cc:299] Negotiation successful
0806 04:57:11.152629 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260806 04:57:11.154852 9601 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu pid 9757
I20260806 04:57:11.166946 10377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:57:11.155989 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37752 (local address 127.9.96.65:33055)
0806 04:57:11.156139 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0806 04:57:11.156143 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 04:57:11.156456 (+ 313us) server_negotiation.cc:407] Connection header received
0806 04:57:11.156563 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:57:11.156566 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:57:11.156616 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:57:11.156688 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:57:11.157792 (+ 1104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:57:11.158532 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:57:11.159760 (+ 1228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:57:11.160156 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:57:11.160248 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:57:11.162284 (+ 2036us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:57:11.162288 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:57:11.162291 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:57:11.162318 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:57:11.164923 (+ 2605us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:57:11.165483 (+ 560us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:57:11.165486 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:57:11.165488 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:57:11.165614 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:57:11.165925 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:57:11.165928 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:57:11.165930 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:57:11.166144 (+ 214us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:57:11.166310 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0806 04:57:11.166559 (+ 249us) server_negotiation.cc:299] Negotiation successful
0806 04:57:11.166696 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260806 04:57:11.167833 9955 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 04:57:11.256317 9601 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu with pid 9757
I20260806 04:57:11.279526 9601 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu pid 9970
Aug 06 04:57:11 dist-test-slave-x4qh krb5kdc[9608](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992231, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.96.66@KRBTEST.COM
I20260806 04:57:11.294801 10388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:57:11.280447 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39620 (local address 127.9.96.66:36881)
0806 04:57:11.280987 (+ 540us) server_negotiation.cc:206] Beginning negotiation
0806 04:57:11.280995 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 04:57:11.281018 (+ 23us) server_negotiation.cc:407] Connection header received
0806 04:57:11.281080 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:57:11.281086 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:57:11.281284 (+ 198us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:57:11.281432 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:57:11.282699 (+ 1267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:57:11.283830 (+ 1131us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:57:11.285032 (+ 1202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:57:11.285553 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:57:11.285656 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:57:11.289536 (+ 3880us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:57:11.289546 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:57:11.289558 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:57:11.289589 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:57:11.292510 (+ 2921us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:57:11.293074 (+ 564us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:57:11.293081 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:57:11.293083 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:57:11.293201 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:57:11.293550 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:57:11.293553 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:57:11.293555 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:57:11.293879 (+ 324us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:57:11.294119 (+ 240us) server_negotiation.cc:1035] Waiting for connection context
0806 04:57:11.294331 (+ 212us) server_negotiation.cc:299] Negotiation successful
0806 04:57:11.294513 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":413,"thread_start_us":270,"threads_started":1}
I20260806 04:57:11.296124 10155 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 04:57:11.379437 9601 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu with pid 9970
I20260806 04:57:11.403251 9601 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu pid 10171
Aug 06 04:57:11 dist-test-slave-x4qh krb5kdc[9608](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992231, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.96.67@KRBTEST.COM
I20260806 04:57:11.419677 10390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:57:11.404339 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33462 (local address 127.9.96.67:37017)
0806 04:57:11.404996 (+ 657us) server_negotiation.cc:206] Beginning negotiation
0806 04:57:11.405003 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 04:57:11.405031 (+ 28us) server_negotiation.cc:407] Connection header received
0806 04:57:11.405091 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:57:11.405097 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:57:11.405311 (+ 214us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:57:11.405488 (+ 177us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:57:11.406808 (+ 1320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:57:11.407963 (+ 1155us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:57:11.409280 (+ 1317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:57:11.409853 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:57:11.410002 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:57:11.414072 (+ 4070us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:57:11.414083 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:57:11.414097 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:57:11.414128 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:57:11.416895 (+ 2767us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:57:11.417536 (+ 641us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:57:11.417543 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:57:11.417546 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:57:11.417665 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:57:11.418051 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:57:11.418057 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:57:11.418060 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:57:11.418381 (+ 321us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:57:11.418722 (+ 341us) server_negotiation.cc:1035] Waiting for connection context
0806 04:57:11.419040 (+ 318us) server_negotiation.cc:299] Negotiation successful
0806 04:57:11.419303 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":501,"thread_start_us":315,"threads_started":1}
I20260806 04:57:11.420957 10360 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 04:57:11.502969 9601 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu with pid 10171
I20260806 04:57:11.526800 9601 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu pid 9627
I20260806 04:57:11.528033 9715 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 04:57:11.627472 9601 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu with pid 9627
2026-08-06T04:57:11Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (4507 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785992227050689-9601-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 04:57:11 dist-test-slave-x4qh krb5kdc[10398](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 04:57:11 dist-test-slave-x4qh krb5kdc[10398](info): set up 2 sockets
Aug 06 04:57:11 dist-test-slave-x4qh krb5kdc[10398](info): commencing operation
krb5kdc: starting...
W20260806 04:57:12.897655 9601 mini_kdc.cc:121] Time spent starting KDC: real 1.217s user 0.001s sys 0.016s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785992227050689-9601-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785992227050689-9601-0/krb5kdc/test-user.keytab.
Aug 06 04:57:13 dist-test-slave-x4qh krb5kdc[10398](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992233, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T04:57:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T04:57:13Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.96.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.96.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.96.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.126.keytab.
Entry for principal kudu/127.9.96.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.126.keytab.
I20260806 04:57:13.079453 9601 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
/tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785992227050689-9601-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785992227050689-9601-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785992227050689-9601-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785992227050689-9601-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.96.126:33081
--webserver_interface=127.9.96.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.126.keytab
--principal=kudu/127.9.96.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.96.84:39649
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.96.126:33081
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785992227050689-9601-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785992227050689-9601-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785992227050689-9601-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 04:57:13.393649 10417 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 04:57:13.393975 10417 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 04:57:13.394142 10417 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 04:57:13.401028 10417 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 04:57:13.401165 10417 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 04:57:13.401271 10417 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 04:57:13.401378 10417 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 04:57:13.401446 10417 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 04:57:13.410748 10417 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.96.84:39649
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785992227050689-9601-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785992227050689-9601-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.96.126:33081
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.126.keytab
--principal=kudu/127.9.96.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.96.126:33081
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785992227050689-9601-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.96.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785992227050689-9601-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 03:56:12 UTC on ccycloud.quasar-guijbm.root.comops.site
build id 4569618
ASAN enabled
I20260806 04:57:13.411983 10417 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 04:57:13.413218 10417 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 04:57:13.424825 10425 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:57:13.424801 10426 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:57:13.424890 10429 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:57:13.425694 10417 server_base.cc:1034] running on GCE node
I20260806 04:57:13.426389 10417 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 04:57:13.427701 10417 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 04:57:13.428937 10417 hybrid_clock.cc:648] HybridClock initialized: now 1785992233428886 us; error 54 us; skew 500 ppm
I20260806 04:57:13.429276 10417 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 04:57:13 dist-test-slave-x4qh krb5kdc[10398](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992233, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.96.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 04:57:13.432328 10417 init.cc:377] Logged in from keytab as kudu/127.9.96.126@KRBTEST.COM (short username kudu)
I20260806 04:57:13.433758 10417 webserver.cc:460] Webserver started at http://127.9.96.126:35923/ using document root <none> and password file <none>
I20260806 04:57:13.434429 10417 fs_manager.cc:359] Metadata directory not provided
I20260806 04:57:13.434578 10417 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 04:57:13.434893 10417 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 04:57:13.437819 10417 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785992227050689-9601-0/minicluster-data/master-0/data/instance:
uuid: "031280f725434213af21051fa2956373"
format_stamp: "Formatted at 2026-08-06 04:57:13 on dist-test-slave-x4qh"
server_key: "86eef449974a36211decd9a1bde6f493"
server_key_iv: "09fe78265b148c06ba4e763700c67107"
server_key_version: "encryptionkey@0"
I20260806 04:57:13.438549 10417 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785992227050689-9601-0/minicluster-data/master-0/wal/instance:
uuid: "031280f725434213af21051fa2956373"
format_stamp: "Formatted at 2026-08-06 04:57:13 on dist-test-slave-x4qh"
server_key: "86eef449974a36211decd9a1bde6f493"
server_key_iv: "09fe78265b148c06ba4e763700c67107"
server_key_version: "encryptionkey@0"
I20260806 04:57:13.443570 10417 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260806 04:57:13.446852 10442 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 04:57:13.448046 10417 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 04:57:13.448230 10417 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785992227050689-9601-0/minicluster-data/master-0/data,/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785992227050689-9601-0/minicluster-data/master-0/wal
uuid: "031280f725434213af21051fa2956373"
format_stamp: "Formatted at 2026-08-06 04:57:13 on dist-test-slave-x4qh"
server_key: "86eef449974a36211decd9a1bde6f493"
server_key_iv: "09fe78265b148c06ba4e763700c67107"
server_key_version: "encryptionkey@0"
I20260806 04:57:13.448426 10417 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785992227050689-9601-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785992227050689-9601-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785992227050689-9601-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 04:57:13.461316 10417 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 04:57:13.465174 10417 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 04:57:13.465461 10417 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 04:57:13.484269 10417 rpc_server.cc:307] RPC server started. Bound to: 127.9.96.126:33081
I20260806 04:57:13.484351 10515 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.96.126:33081 every 8 connection(s)
I20260806 04:57:13.485594 10417 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785992227050689-9601-0/minicluster-data/master-0/data/info.pb
I20260806 04:57:13.489100 10516 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 04:57:13.494172 9601 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu as pid 10417
I20260806 04:57:13.494415 9601 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785992227050689-9601-0/minicluster-data/master-0/wal/instance
I20260806 04:57:13.495285 9601 external_mini_cluster.cc:1428] Setting key acc4de63bd601c0b37c6f38b97ccdeb9
I20260806 04:57:13.500146 10516 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 031280f725434213af21051fa2956373: Bootstrap starting.
I20260806 04:57:13.503700 10516 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 031280f725434213af21051fa2956373: Neither blocks nor log segments found. Creating new log.
I20260806 04:57:13.504966 10516 log.cc:824] T 00000000000000000000000000000000 P 031280f725434213af21051fa2956373: Log is configured to *not* fsync() on all Append() calls
I20260806 04:57:13.508250 10516 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 031280f725434213af21051fa2956373: No bootstrap required, opened a new log
Aug 06 04:57:13 dist-test-slave-x4qh krb5kdc[10398](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992233, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.96.126@KRBTEST.COM
I20260806 04:57:13.518321 10516 raft_consensus.cc:348] T 00000000000000000000000000000000 P 031280f725434213af21051fa2956373 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "031280f725434213af21051fa2956373" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 33081 } }
I20260806 04:57:13.518699 10516 raft_consensus.cc:374] T 00000000000000000000000000000000 P 031280f725434213af21051fa2956373 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 04:57:13.518812 10516 raft_consensus.cc:729] T 00000000000000000000000000000000 P 031280f725434213af21051fa2956373 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 031280f725434213af21051fa2956373, State: Initialized, Role: FOLLOWER
I20260806 04:57:13.519351 10516 consensus_queue.cc:260] T 00000000000000000000000000000000 P 031280f725434213af21051fa2956373 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "031280f725434213af21051fa2956373" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 33081 } }
I20260806 04:57:13.519681 10516 raft_consensus.cc:388] T 00000000000000000000000000000000 P 031280f725434213af21051fa2956373 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 04:57:13.519853 10516 raft_consensus.cc:482] T 00000000000000000000000000000000 P 031280f725434213af21051fa2956373 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 04:57:13.520056 10516 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 031280f725434213af21051fa2956373 [term 0 FOLLOWER]: Advancing to term 1
I20260806 04:57:13.521045 10519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:57:13.497328 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39496 (local address 127.9.96.126:33081)
0806 04:57:13.498176 (+ 848us) server_negotiation.cc:206] Beginning negotiation
0806 04:57:13.498188 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 04:57:13.498228 (+ 40us) server_negotiation.cc:407] Connection header received
0806 04:57:13.499974 (+ 1746us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:57:13.499994 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:57:13.500363 (+ 369us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:57:13.500657 (+ 294us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:57:13.502498 (+ 1841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:57:13.503818 (+ 1320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:57:13.505698 (+ 1880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:57:13.506794 (+ 1096us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:57:13.506978 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:57:13.512409 (+ 5431us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:57:13.512421 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:57:13.512438 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:57:13.512486 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:57:13.516822 (+ 4336us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:57:13.517603 (+ 781us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:57:13.517609 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:57:13.517615 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:57:13.517769 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:57:13.518187 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:57:13.518190 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:57:13.518192 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:57:13.518715 (+ 523us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:57:13.519010 (+ 295us) server_negotiation.cc:1035] Waiting for connection context
0806 04:57:13.519566 (+ 556us) server_negotiation.cc:299] Negotiation successful
0806 04:57:13.519963 (+ 397us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":646,"thread_start_us":412,"threads_started":1}
I20260806 04:57:13.522603 10516 raft_consensus.cc:504] T 00000000000000000000000000000000 P 031280f725434213af21051fa2956373 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "031280f725434213af21051fa2956373" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 33081 } }
I20260806 04:57:13.523160 10516 leader_election.cc:302] T 00000000000000000000000000000000 P 031280f725434213af21051fa2956373 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 031280f725434213af21051fa2956373; no voters:
I20260806 04:57:13.523860 10516 leader_election.cc:288] T 00000000000000000000000000000000 P 031280f725434213af21051fa2956373 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 04:57:13.524052 10521 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 031280f725434213af21051fa2956373 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 04:57:13.524796 10521 raft_consensus.cc:686] T 00000000000000000000000000000000 P 031280f725434213af21051fa2956373 [term 1 LEADER]: Becoming Leader. State: Replica: 031280f725434213af21051fa2956373, State: Running, Role: LEADER
I20260806 04:57:13.525259 10521 consensus_queue.cc:237] T 00000000000000000000000000000000 P 031280f725434213af21051fa2956373 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "031280f725434213af21051fa2956373" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 33081 } }
I20260806 04:57:13.525887 10516 sys_catalog.cc:564] T 00000000000000000000000000000000 P 031280f725434213af21051fa2956373 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 04:57:13.528427 10523 sys_catalog.cc:455] T 00000000000000000000000000000000 P 031280f725434213af21051fa2956373 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 031280f725434213af21051fa2956373. Latest consensus state: current_term: 1 leader_uuid: "031280f725434213af21051fa2956373" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "031280f725434213af21051fa2956373" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 33081 } } }
I20260806 04:57:13.528939 10523 sys_catalog.cc:458] T 00000000000000000000000000000000 P 031280f725434213af21051fa2956373 [sys.catalog]: This master's current role is: LEADER
I20260806 04:57:13.529203 10522 sys_catalog.cc:455] T 00000000000000000000000000000000 P 031280f725434213af21051fa2956373 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "031280f725434213af21051fa2956373" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "031280f725434213af21051fa2956373" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 33081 } } }
I20260806 04:57:13.529649 10522 sys_catalog.cc:458] T 00000000000000000000000000000000 P 031280f725434213af21051fa2956373 [sys.catalog]: This master's current role is: LEADER
I20260806 04:57:13.532778 10529 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 04:57:13.538925 10529 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 04:57:13.548094 10529 catalog_manager.cc:1347] Generated new cluster ID: 790a23babda548068b82bdfc53fea14e
I20260806 04:57:13.548195 10529 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 04:57:13.566094 10529 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 04:57:13.567838 10529 catalog_manager.cc:1504] Loading token signing keys...
I20260806 04:57:13.575255 10529 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 031280f725434213af21051fa2956373: Generated new TSK 0
I20260806 04:57:13.576148 10529 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.9.96.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.96.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.96.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.65.keytab.
Entry for principal kudu/127.9.96.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.65.keytab.
I20260806 04:57:13.629338 9601 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
/tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785992227050689-9601-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785992227050689-9601-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785992227050689-9601-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785992227050689-9601-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.96.65:0
--local_ip_for_outbound_sockets=127.9.96.65
--webserver_interface=127.9.96.65
--webserver_port=0
--tserver_master_addrs=127.9.96.126:33081
--keytab_file=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.65.keytab
--principal=kudu/127.9.96.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.96.84:39649
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785992227050689-9601-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785992227050689-9601-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785992227050689-9601-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 04:57:13.951099 10552 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 04:57:13.951529 10552 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 04:57:13.951759 10552 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 04:57:13.959331 10552 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 04:57:13.959537 10552 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 04:57:13.959758 10552 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.96.65
I20260806 04:57:13.969683 10552 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.96.84:39649
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785992227050689-9601-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785992227050689-9601-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.65.keytab
--principal=kudu/127.9.96.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.96.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785992227050689-9601-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.96.65
--webserver_port=0
--tserver_master_addrs=127.9.96.126:33081
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.96.65
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785992227050689-9601-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 03:56:12 UTC on ccycloud.quasar-guijbm.root.comops.site
build id 4569618
ASAN enabled
I20260806 04:57:13.970966 10552 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 04:57:13.972379 10552 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 04:57:13.986290 10561 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:57:13.986240 10564 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:57:13.986368 10562 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:57:13.992847 10552 server_base.cc:1034] running on GCE node
I20260806 04:57:13.993461 10552 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 04:57:13.994421 10552 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 04:57:13.995647 10552 hybrid_clock.cc:648] HybridClock initialized: now 1785992233995614 us; error 56 us; skew 500 ppm
I20260806 04:57:13.996012 10552 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 04:57:13 dist-test-slave-x4qh krb5kdc[10398](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992233, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.96.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 04:57:13.999203 10552 init.cc:377] Logged in from keytab as kudu/127.9.96.65@KRBTEST.COM (short username kudu)
I20260806 04:57:14.000690 10552 webserver.cc:460] Webserver started at http://127.9.96.65:37581/ using document root <none> and password file <none>
I20260806 04:57:14.001425 10552 fs_manager.cc:359] Metadata directory not provided
I20260806 04:57:14.001569 10552 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 04:57:14.001883 10552 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 04:57:14.004979 10552 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785992227050689-9601-0/minicluster-data/ts-0/data/instance:
uuid: "dcbd463d6deb462f960ac097fc3ebfea"
format_stamp: "Formatted at 2026-08-06 04:57:13 on dist-test-slave-x4qh"
server_key: "6599efd5f25de9df54eef2d1776b9be1"
server_key_iv: "f26511f1ee44ef054b898578f62639aa"
server_key_version: "encryptionkey@0"
I20260806 04:57:14.005784 10552 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785992227050689-9601-0/minicluster-data/ts-0/wal/instance:
uuid: "dcbd463d6deb462f960ac097fc3ebfea"
format_stamp: "Formatted at 2026-08-06 04:57:13 on dist-test-slave-x4qh"
server_key: "6599efd5f25de9df54eef2d1776b9be1"
server_key_iv: "f26511f1ee44ef054b898578f62639aa"
server_key_version: "encryptionkey@0"
I20260806 04:57:14.011180 10552 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260806 04:57:14.014597 10572 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 04:57:14.015882 10552 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 04:57:14.016055 10552 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785992227050689-9601-0/minicluster-data/ts-0/data,/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785992227050689-9601-0/minicluster-data/ts-0/wal
uuid: "dcbd463d6deb462f960ac097fc3ebfea"
format_stamp: "Formatted at 2026-08-06 04:57:13 on dist-test-slave-x4qh"
server_key: "6599efd5f25de9df54eef2d1776b9be1"
server_key_iv: "f26511f1ee44ef054b898578f62639aa"
server_key_version: "encryptionkey@0"
I20260806 04:57:14.016285 10552 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785992227050689-9601-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785992227050689-9601-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785992227050689-9601-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 04:57:14.033586 10552 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 04:57:14.037503 10552 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 04:57:14.037803 10552 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 04:57:14.038750 10552 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 04:57:14.040060 10552 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 04:57:14.040166 10552 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 04:57:14.040307 10552 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 04:57:14.040385 10552 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 04:57:14.075186 10552 rpc_server.cc:307] RPC server started. Bound to: 127.9.96.65:46643
I20260806 04:57:14.075286 10727 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.96.65:46643 every 8 connection(s)
I20260806 04:57:14.076738 10552 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785992227050689-9601-0/minicluster-data/ts-0/data/info.pb
I20260806 04:57:14.078943 9601 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu as pid 10552
I20260806 04:57:14.079190 9601 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785992227050689-9601-0/minicluster-data/ts-0/wal/instance
I20260806 04:57:14.080027 9601 external_mini_cluster.cc:1428] Setting key 4fb3c5ffd877c3f57ec4d8fb5d41b1cb
Aug 06 04:57:14 dist-test-slave-x4qh krb5kdc[10398](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992233, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.96.65@KRBTEST.COM for kudu/127.9.96.126@KRBTEST.COM
I20260806 04:57:14.101028 10731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:57:14.079711 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.96.65:48781 (local address 127.9.96.126:33081)
0806 04:57:14.080230 (+ 519us) server_negotiation.cc:206] Beginning negotiation
0806 04:57:14.080237 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 04:57:14.081401 (+ 1164us) server_negotiation.cc:407] Connection header received
0806 04:57:14.084012 (+ 2611us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:57:14.084016 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:57:14.084089 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:57:14.084198 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:57:14.086400 (+ 2202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:57:14.087192 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:57:14.089351 (+ 2159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:57:14.090067 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:57:14.090209 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:57:14.093932 (+ 3723us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:57:14.093943 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:57:14.093952 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:57:14.094005 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:57:14.097498 (+ 3493us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:57:14.098535 (+ 1037us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:57:14.098539 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:57:14.098541 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:57:14.098658 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:57:14.099191 (+ 533us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:57:14.099196 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:57:14.099199 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:57:14.099524 (+ 325us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:57:14.099644 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0806 04:57:14.100562 (+ 918us) server_negotiation.cc:299] Negotiation successful
0806 04:57:14.100755 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":368,"thread_start_us":239,"threads_started":1}
WARNING: no policy specified for kudu/127.9.96.66@KRBTEST.COM; defaulting to no policy
I20260806 04:57:14.102341 10732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:57:14.080348 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:33081 (local address 127.9.96.65:48781)
0806 04:57:14.081214 (+ 866us) negotiation.cc:107] Waiting for socket to connect
0806 04:57:14.081268 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0806 04:57:14.083683 (+ 2415us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:57:14.084515 (+ 832us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:57:14.084529 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:57:14.085370 (+ 841us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:57:14.086195 (+ 825us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:57:14.086224 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:57:14.087364 (+ 1140us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:57:14.087395 (+ 31us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:57:14.089194 (+ 1799us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:57:14.089207 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:57:14.090299 (+ 1092us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:57:14.090304 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:57:14.090499 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:57:14.091222 (+ 723us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:57:14.091248 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:57:14.093689 (+ 2441us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 04:57:14.097644 (+ 3955us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:57:14.097653 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:57:14.097672 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:57:14.098361 (+ 689us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:57:14.098805 (+ 444us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:57:14.098810 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:57:14.098815 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:57:14.099069 (+ 254us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:57:14.099670 (+ 601us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 04:57:14.099698 (+ 28us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 04:57:14.100127 (+ 429us) client_negotiation.cc:769] Sending connection context
0806 04:57:14.100483 (+ 356us) client_negotiation.cc:240] Negotiation successful
0806 04:57:14.101121 (+ 638us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":574,"thread_start_us":344,"threads_started":1}
I20260806 04:57:14.104223 10728 heartbeater.cc:347] Connected to a master server at 127.9.96.126:33081
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.96.66@KRBTEST.COM" created.
I20260806 04:57:14.104532 10728 heartbeater.cc:464] Registering TS with master...
I20260806 04:57:14.105237 10728 heartbeater.cc:511] Master 127.9.96.126:33081 requested a full tablet report, sending...
I20260806 04:57:14.107275 10468 ts_manager.cc:194] Registered new tserver with Master: dcbd463d6deb462f960ac097fc3ebfea (127.9.96.65:46643)
I20260806 04:57:14.108737 10468 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.96.65@KRBTEST.COM'} at 127.9.96.65:48781
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.96.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.66.keytab.
Entry for principal kudu/127.9.96.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.66.keytab.
I20260806 04:57:14.127738 9601 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
/tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785992227050689-9601-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785992227050689-9601-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785992227050689-9601-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785992227050689-9601-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.96.66:0
--local_ip_for_outbound_sockets=127.9.96.66
--webserver_interface=127.9.96.66
--webserver_port=0
--tserver_master_addrs=127.9.96.126:33081
--keytab_file=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.66.keytab
--principal=kudu/127.9.96.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.96.84:39649
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785992227050689-9601-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785992227050689-9601-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785992227050689-9601-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 04:57:14.447536 10735 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 04:57:14.447798 10735 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 04:57:14.447956 10735 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 04:57:14.455031 10735 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 04:57:14.455130 10735 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 04:57:14.455340 10735 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.96.66
I20260806 04:57:14.464612 10735 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.96.84:39649
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785992227050689-9601-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785992227050689-9601-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.66.keytab
--principal=kudu/127.9.96.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.96.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785992227050689-9601-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.96.66
--webserver_port=0
--tserver_master_addrs=127.9.96.126:33081
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.96.66
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785992227050689-9601-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 03:56:12 UTC on ccycloud.quasar-guijbm.root.comops.site
build id 4569618
ASAN enabled
I20260806 04:57:14.465760 10735 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 04:57:14.467011 10735 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 04:57:14.480171 10743 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:57:14.480222 10747 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:57:14.480189 10749 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:57:14.480959 10735 server_base.cc:1034] running on GCE node
I20260806 04:57:14.481563 10735 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 04:57:14.482383 10735 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 04:57:14.483614 10735 hybrid_clock.cc:648] HybridClock initialized: now 1785992234483570 us; error 53 us; skew 500 ppm
I20260806 04:57:14.483960 10735 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 04:57:14 dist-test-slave-x4qh krb5kdc[10398](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992234, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.96.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 04:57:14.487171 10735 init.cc:377] Logged in from keytab as kudu/127.9.96.66@KRBTEST.COM (short username kudu)
I20260806 04:57:14.488653 10735 webserver.cc:460] Webserver started at http://127.9.96.66:34763/ using document root <none> and password file <none>
I20260806 04:57:14.489316 10735 fs_manager.cc:359] Metadata directory not provided
I20260806 04:57:14.489461 10735 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 04:57:14.489790 10735 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 04:57:14.492522 10735 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785992227050689-9601-0/minicluster-data/ts-1/data/instance:
uuid: "4922b622110d494389f3b220c71ca9e1"
format_stamp: "Formatted at 2026-08-06 04:57:14 on dist-test-slave-x4qh"
server_key: "f06cc6b65472499e30325ae3aa6db221"
server_key_iv: "d299262ec0645bc227e0ef1caafa3c95"
server_key_version: "encryptionkey@0"
I20260806 04:57:14.493332 10735 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785992227050689-9601-0/minicluster-data/ts-1/wal/instance:
uuid: "4922b622110d494389f3b220c71ca9e1"
format_stamp: "Formatted at 2026-08-06 04:57:14 on dist-test-slave-x4qh"
server_key: "f06cc6b65472499e30325ae3aa6db221"
server_key_iv: "d299262ec0645bc227e0ef1caafa3c95"
server_key_version: "encryptionkey@0"
I20260806 04:57:14.498335 10735 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.005s
I20260806 04:57:14.501638 10760 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 04:57:14.503027 10735 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 04:57:14.503211 10735 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785992227050689-9601-0/minicluster-data/ts-1/data,/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785992227050689-9601-0/minicluster-data/ts-1/wal
uuid: "4922b622110d494389f3b220c71ca9e1"
format_stamp: "Formatted at 2026-08-06 04:57:14 on dist-test-slave-x4qh"
server_key: "f06cc6b65472499e30325ae3aa6db221"
server_key_iv: "d299262ec0645bc227e0ef1caafa3c95"
server_key_version: "encryptionkey@0"
I20260806 04:57:14.503443 10735 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785992227050689-9601-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785992227050689-9601-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785992227050689-9601-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 04:57:14.525049 10735 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 04:57:14.529615 10735 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 04:57:14.529917 10735 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 04:57:14.530889 10735 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 04:57:14.532308 10735 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 04:57:14.532424 10735 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 04:57:14.532586 10735 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 04:57:14.532660 10735 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 04:57:14.566980 10735 rpc_server.cc:307] RPC server started. Bound to: 127.9.96.66:41241
I20260806 04:57:14.567030 10929 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.96.66:41241 every 8 connection(s)
I20260806 04:57:14.568327 10735 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785992227050689-9601-0/minicluster-data/ts-1/data/info.pb
I20260806 04:57:14.574240 9601 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu as pid 10735
I20260806 04:57:14.574452 9601 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785992227050689-9601-0/minicluster-data/ts-1/wal/instance
I20260806 04:57:14.574896 9601 external_mini_cluster.cc:1428] Setting key da46ec9c7e5863b41a1870c98047980b
Aug 06 04:57:14 dist-test-slave-x4qh krb5kdc[10398](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992234, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.96.66@KRBTEST.COM for kudu/127.9.96.126@KRBTEST.COM
I20260806 04:57:14.589025 10731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:57:14.571015 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.96.66:45929 (local address 127.9.96.126:33081)
0806 04:57:14.571223 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0806 04:57:14.571228 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 04:57:14.572421 (+ 1193us) server_negotiation.cc:407] Connection header received
0806 04:57:14.574195 (+ 1774us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:57:14.574201 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:57:14.574296 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:57:14.574453 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:57:14.576210 (+ 1757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:57:14.576931 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:57:14.578478 (+ 1547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:57:14.578897 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:57:14.579042 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:57:14.582829 (+ 3787us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:57:14.582833 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:57:14.582837 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:57:14.582870 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:57:14.585579 (+ 2709us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:57:14.586537 (+ 958us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:57:14.586540 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:57:14.586543 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:57:14.586672 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:57:14.587215 (+ 543us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:57:14.587219 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:57:14.587221 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:57:14.587553 (+ 332us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:57:14.587664 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0806 04:57:14.588610 (+ 946us) server_negotiation.cc:299] Negotiation successful
0806 04:57:14.588785 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
I20260806 04:57:14.590631 10933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:57:14.571537 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:33081 (local address 127.9.96.66:45929)
0806 04:57:14.572261 (+ 724us) negotiation.cc:107] Waiting for socket to connect
0806 04:57:14.572299 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0806 04:57:14.573928 (+ 1629us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:57:14.574709 (+ 781us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:57:14.574717 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:57:14.575338 (+ 621us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:57:14.576026 (+ 688us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:57:14.576043 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:57:14.577098 (+ 1055us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:57:14.577102 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:57:14.578319 (+ 1217us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:57:14.578331 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:57:14.579128 (+ 797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:57:14.579135 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:57:14.579358 (+ 223us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:57:14.580118 (+ 760us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:57:14.580140 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:57:14.582574 (+ 2434us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 04:57:14.585726 (+ 3152us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:57:14.585738 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:57:14.585756 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:57:14.586411 (+ 655us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:57:14.586811 (+ 400us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:57:14.586818 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:57:14.586825 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:57:14.587089 (+ 264us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:57:14.587679 (+ 590us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 04:57:14.587690 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 04:57:14.588158 (+ 468us) client_negotiation.cc:769] Sending connection context
0806 04:57:14.588510 (+ 352us) client_negotiation.cc:240] Negotiation successful
0806 04:57:14.589068 (+ 558us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":501,"thread_start_us":291,"threads_started":1}
I20260806 04:57:14.592223 10930 heartbeater.cc:347] Connected to a master server at 127.9.96.126:33081
I20260806 04:57:14.592532 10930 heartbeater.cc:464] Registering TS with master...
I20260806 04:57:14.593262 10930 heartbeater.cc:511] Master 127.9.96.126:33081 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.9.96.67@KRBTEST.COM; defaulting to no policy
I20260806 04:57:14.594769 10468 ts_manager.cc:194] Registered new tserver with Master: 4922b622110d494389f3b220c71ca9e1 (127.9.96.66:41241)
I20260806 04:57:14.595609 10468 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.96.66@KRBTEST.COM'} at 127.9.96.66:45929
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.96.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.96.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.67.keytab.
Entry for principal kudu/127.9.96.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.67.keytab.
I20260806 04:57:14.618546 9601 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
/tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785992227050689-9601-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785992227050689-9601-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785992227050689-9601-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785992227050689-9601-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.96.67:0
--local_ip_for_outbound_sockets=127.9.96.67
--webserver_interface=127.9.96.67
--webserver_port=0
--tserver_master_addrs=127.9.96.126:33081
--keytab_file=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.67.keytab
--principal=kudu/127.9.96.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.96.84:39649
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785992227050689-9601-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785992227050689-9601-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785992227050689-9601-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 04:57:14.937858 10940 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 04:57:14.938162 10940 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 04:57:14.938335 10940 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 04:57:14.945010 10940 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 04:57:14.945153 10940 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 04:57:14.945369 10940 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.96.67
I20260806 04:57:14.954962 10940 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.96.84:39649
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785992227050689-9601-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785992227050689-9601-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.67.keytab
--principal=kudu/127.9.96.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.96.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785992227050689-9601-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.96.67
--webserver_port=0
--tserver_master_addrs=127.9.96.126:33081
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.96.67
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785992227050689-9601-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 03:56:12 UTC on ccycloud.quasar-guijbm.root.comops.site
build id 4569618
ASAN enabled
I20260806 04:57:14.956154 10940 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 04:57:14.957513 10940 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 04:57:14.970289 10947 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:57:14.970561 10949 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:57:14.970963 10951 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:57:14.971902 10940 server_base.cc:1034] running on GCE node
I20260806 04:57:14.972404 10940 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 04:57:14.973213 10940 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 04:57:14.974429 10940 hybrid_clock.cc:648] HybridClock initialized: now 1785992234974376 us; error 29 us; skew 500 ppm
I20260806 04:57:14.974747 10940 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 04:57:14 dist-test-slave-x4qh krb5kdc[10398](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992234, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.96.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 04:57:14.977893 10940 init.cc:377] Logged in from keytab as kudu/127.9.96.67@KRBTEST.COM (short username kudu)
I20260806 04:57:14.979247 10940 webserver.cc:460] Webserver started at http://127.9.96.67:39197/ using document root <none> and password file <none>
I20260806 04:57:14.979964 10940 fs_manager.cc:359] Metadata directory not provided
I20260806 04:57:14.980064 10940 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 04:57:14.980324 10940 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 04:57:14.983129 10940 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785992227050689-9601-0/minicluster-data/ts-2/data/instance:
uuid: "71b0e7603bf149f0a526d89886013a38"
format_stamp: "Formatted at 2026-08-06 04:57:14 on dist-test-slave-x4qh"
server_key: "0813567f0ef65c79f2705f353ee4721a"
server_key_iv: "9a89d80826a6edea96995609e6c200d2"
server_key_version: "encryptionkey@0"
I20260806 04:57:14.983959 10940 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785992227050689-9601-0/minicluster-data/ts-2/wal/instance:
uuid: "71b0e7603bf149f0a526d89886013a38"
format_stamp: "Formatted at 2026-08-06 04:57:14 on dist-test-slave-x4qh"
server_key: "0813567f0ef65c79f2705f353ee4721a"
server_key_iv: "9a89d80826a6edea96995609e6c200d2"
server_key_version: "encryptionkey@0"
I20260806 04:57:14.988840 10940 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260806 04:57:14.992273 10959 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 04:57:14.993523 10940 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 04:57:14.993688 10940 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785992227050689-9601-0/minicluster-data/ts-2/data,/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785992227050689-9601-0/minicluster-data/ts-2/wal
uuid: "71b0e7603bf149f0a526d89886013a38"
format_stamp: "Formatted at 2026-08-06 04:57:14 on dist-test-slave-x4qh"
server_key: "0813567f0ef65c79f2705f353ee4721a"
server_key_iv: "9a89d80826a6edea96995609e6c200d2"
server_key_version: "encryptionkey@0"
I20260806 04:57:14.993853 10940 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785992227050689-9601-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785992227050689-9601-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785992227050689-9601-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 04:57:15.015829 10940 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 04:57:15.020744 10940 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 04:57:15.021106 10940 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 04:57:15.022217 10940 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 04:57:15.023722 10940 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 04:57:15.023824 10940 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 04:57:15.024004 10940 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 04:57:15.024060 10940 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 04:57:15.061439 10940 rpc_server.cc:307] RPC server started. Bound to: 127.9.96.67:45891
I20260806 04:57:15.061482 11125 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.96.67:45891 every 8 connection(s)
I20260806 04:57:15.062798 10940 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785992227050689-9601-0/minicluster-data/ts-2/data/info.pb
I20260806 04:57:15.064733 9601 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu as pid 10940
I20260806 04:57:15.064961 9601 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785992227050689-9601-0/minicluster-data/ts-2/wal/instance
I20260806 04:57:15.065428 9601 external_mini_cluster.cc:1428] Setting key 22397c5524dc7653d85a751f14ce5830
Aug 06 04:57:15 dist-test-slave-x4qh krb5kdc[10398](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992234, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.96.67@KRBTEST.COM for kudu/127.9.96.126@KRBTEST.COM
I20260806 04:57:15.084282 10731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:57:15.065585 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.96.67:35265 (local address 127.9.96.126:33081)
0806 04:57:15.065846 (+ 261us) server_negotiation.cc:206] Beginning negotiation
0806 04:57:15.065853 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 04:57:15.067517 (+ 1664us) server_negotiation.cc:407] Connection header received
0806 04:57:15.069545 (+ 2028us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:57:15.069551 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:57:15.069676 (+ 125us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:57:15.069796 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:57:15.071257 (+ 1461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:57:15.072074 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:57:15.073636 (+ 1562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:57:15.074042 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:57:15.074214 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:57:15.077847 (+ 3633us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:57:15.077852 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:57:15.077855 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:57:15.077886 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:57:15.081721 (+ 3835us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:57:15.082327 (+ 606us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:57:15.082330 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:57:15.082332 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:57:15.082463 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:57:15.082823 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:57:15.082826 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:57:15.082828 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:57:15.083076 (+ 248us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:57:15.083181 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0806 04:57:15.083867 (+ 686us) server_negotiation.cc:299] Negotiation successful
0806 04:57:15.084017 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":102}
I20260806 04:57:15.085188 11130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:57:15.066310 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:33081 (local address 127.9.96.67:35265)
0806 04:57:15.067328 (+ 1018us) negotiation.cc:107] Waiting for socket to connect
0806 04:57:15.067406 (+ 78us) client_negotiation.cc:174] Beginning negotiation
0806 04:57:15.069293 (+ 1887us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:57:15.070024 (+ 731us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:57:15.070032 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:57:15.070535 (+ 503us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:57:15.071078 (+ 543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:57:15.071094 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:57:15.072245 (+ 1151us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:57:15.072250 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:57:15.073483 (+ 1233us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:57:15.073500 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:57:15.074278 (+ 778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:57:15.074288 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:57:15.074551 (+ 263us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:57:15.075263 (+ 712us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:57:15.075285 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:57:15.077629 (+ 2344us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 04:57:15.081837 (+ 4208us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:57:15.081843 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:57:15.081853 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:57:15.082229 (+ 376us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:57:15.082554 (+ 325us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:57:15.082556 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:57:15.082559 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:57:15.082733 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:57:15.083198 (+ 465us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 04:57:15.083204 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 04:57:15.083518 (+ 314us) client_negotiation.cc:769] Sending connection context
0806 04:57:15.083765 (+ 247us) client_negotiation.cc:240] Negotiation successful
0806 04:57:15.084114 (+ 349us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":774,"thread_start_us":510,"threads_started":1}
I20260806 04:57:15.086380 11126 heartbeater.cc:347] Connected to a master server at 127.9.96.126:33081
I20260806 04:57:15.086619 11126 heartbeater.cc:464] Registering TS with master...
I20260806 04:57:15.087216 11126 heartbeater.cc:511] Master 127.9.96.126:33081 requested a full tablet report, sending...
I20260806 04:57:15.088649 10468 ts_manager.cc:194] Registered new tserver with Master: 71b0e7603bf149f0a526d89886013a38 (127.9.96.67:45891)
I20260806 04:57:15.089412 10468 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.96.67@KRBTEST.COM'} at 127.9.96.67:35265
I20260806 04:57:15.093444 9601 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 04:57:15.112221 10728 heartbeater.cc:503] Master 127.9.96.126:33081 was elected leader, sending a full tablet report...
I20260806 04:57:15.112775 10731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:57:15.100617 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39502 (local address 127.9.96.126:33081)
0806 04:57:15.100759 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0806 04:57:15.100763 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 04:57:15.101263 (+ 500us) server_negotiation.cc:407] Connection header received
0806 04:57:15.101453 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:57:15.101457 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:57:15.101510 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:57:15.101627 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:57:15.102973 (+ 1346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:57:15.103719 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:57:15.104999 (+ 1280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:57:15.105418 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:57:15.105559 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:57:15.107900 (+ 2341us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:57:15.107904 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:57:15.107907 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:57:15.107954 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:57:15.110198 (+ 2244us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:57:15.110772 (+ 574us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:57:15.110778 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:57:15.110781 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:57:15.110920 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:57:15.111277 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:57:15.111284 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:57:15.111287 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:57:15.111639 (+ 352us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:57:15.111885 (+ 246us) server_negotiation.cc:1035] Waiting for connection context
0806 04:57:15.112237 (+ 352us) server_negotiation.cc:299] Negotiation successful
0806 04:57:15.112440 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260806 04:57:15.120278 10467 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39502:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 04:57:15.128161 10467 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 04:57:15.156543 11144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:57:15.142472 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.65:46643 (local address 127.0.0.1:55862)
0806 04:57:15.143498 (+ 1026us) negotiation.cc:107] Waiting for socket to connect
0806 04:57:15.143555 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0806 04:57:15.143754 (+ 199us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:57:15.148951 (+ 5197us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:57:15.148962 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:57:15.149006 (+ 44us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 04:57:15.149537 (+ 531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:57:15.149553 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:57:15.151603 (+ 2050us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:57:15.151611 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:57:15.153493 (+ 1882us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:57:15.153506 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:57:15.155446 (+ 1940us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:57:15.155453 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:57:15.155658 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:57:15.155707 (+ 49us) client_negotiation.cc:769] Sending connection context
0806 04:57:15.155826 (+ 119us) client_negotiation.cc:240] Negotiation successful
0806 04:57:15.156028 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":694,"mutex_wait_us":28,"thread_start_us":7403,"threads_started":1}
I20260806 04:57:15.158939 11145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:57:15.142725 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55862 (local address 127.9.96.65:46643)
0806 04:57:15.148438 (+ 5713us) server_negotiation.cc:206] Beginning negotiation
0806 04:57:15.148444 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 04:57:15.148467 (+ 23us) server_negotiation.cc:407] Connection header received
0806 04:57:15.148541 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:57:15.148547 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:57:15.148722 (+ 175us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:57:15.148880 (+ 158us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 04:57:15.149751 (+ 871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:57:15.151331 (+ 1580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:57:15.153659 (+ 2328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:57:15.155181 (+ 1522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:57:15.156676 (+ 1495us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:57:15.158499 (+ 1823us) server_negotiation.cc:1035] Waiting for connection context
0806 04:57:15.158580 (+ 81us) server_negotiation.cc:299] Negotiation successful
0806 04:57:15.158710 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5573,"thread_start_us":5414,"threads_started":1}
I20260806 04:57:15.162667 11146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:57:15.143088 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38586 (local address 127.9.96.66:41241)
0806 04:57:15.147933 (+ 4845us) server_negotiation.cc:206] Beginning negotiation
0806 04:57:15.147945 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 04:57:15.150241 (+ 2296us) server_negotiation.cc:407] Connection header received
0806 04:57:15.150614 (+ 373us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:57:15.150622 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:57:15.150840 (+ 218us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:57:15.151029 (+ 189us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 04:57:15.154277 (+ 3248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:57:15.155853 (+ 1576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:57:15.159621 (+ 3768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:57:15.161091 (+ 1470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:57:15.161229 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:57:15.161459 (+ 230us) server_negotiation.cc:1035] Waiting for connection context
0806 04:57:15.162163 (+ 704us) server_negotiation.cc:299] Negotiation successful
0806 04:57:15.162339 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4617,"thread_start_us":4480,"threads_started":1}
I20260806 04:57:15.163421 11143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:57:15.142690 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.66:41241 (local address 127.0.0.1:38586)
0806 04:57:15.150134 (+ 7444us) negotiation.cc:107] Waiting for socket to connect
0806 04:57:15.150153 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0806 04:57:15.150355 (+ 202us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:57:15.151272 (+ 917us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:57:15.151277 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:57:15.151304 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 04:57:15.154054 (+ 2750us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:57:15.154069 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:57:15.156023 (+ 1954us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:57:15.156028 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:57:15.158348 (+ 2320us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:57:15.158373 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:57:15.161673 (+ 3300us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:57:15.161699 (+ 26us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:57:15.161896 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:57:15.161934 (+ 38us) client_negotiation.cc:769] Sending connection context
0806 04:57:15.162938 (+ 1004us) client_negotiation.cc:240] Negotiation successful
0806 04:57:15.163089 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7305,"thread_start_us":555,"threads_started":1}
I20260806 04:57:15.163060 10637 tablet_service.cc:1467] Processing CreateTablet for tablet cc8e3bd8cf894e9c867f2050474a45c7 (DEFAULT_TABLE table=test-table [id=a6e349c196734706866389ce61577c22]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 04:57:15.164846 10637 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cc8e3bd8cf894e9c867f2050474a45c7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 04:57:15.166538 11144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:57:15.150660 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.67:45891 (local address 127.0.0.1:40100)
0806 04:57:15.157118 (+ 6458us) negotiation.cc:107] Waiting for socket to connect
0806 04:57:15.157141 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0806 04:57:15.157232 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:57:15.159242 (+ 2010us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:57:15.159247 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:57:15.159274 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 04:57:15.159446 (+ 172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:57:15.159458 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:57:15.161391 (+ 1933us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:57:15.161396 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:57:15.163199 (+ 1803us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:57:15.163216 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:57:15.165782 (+ 2566us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:57:15.165789 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:57:15.165990 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:57:15.166027 (+ 37us) client_negotiation.cc:769] Sending connection context
0806 04:57:15.166103 (+ 76us) client_negotiation.cc:240] Negotiation successful
0806 04:57:15.166252 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6363,"thread_start_us":6758,"threads_started":1}
I20260806 04:57:15.167021 10831 tablet_service.cc:1467] Processing CreateTablet for tablet cc8e3bd8cf894e9c867f2050474a45c7 (DEFAULT_TABLE table=test-table [id=a6e349c196734706866389ce61577c22]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 04:57:15.167733 11148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:57:15.150803 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40100 (local address 127.9.96.67:45891)
0806 04:57:15.158703 (+ 7900us) server_negotiation.cc:206] Beginning negotiation
0806 04:57:15.158709 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 04:57:15.158737 (+ 28us) server_negotiation.cc:407] Connection header received
0806 04:57:15.158811 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:57:15.158816 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:57:15.158967 (+ 151us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:57:15.159234 (+ 267us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 04:57:15.159621 (+ 387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:57:15.161192 (+ 1571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:57:15.163365 (+ 2173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:57:15.165562 (+ 2197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:57:15.166920 (+ 1358us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:57:15.167134 (+ 214us) server_negotiation.cc:1035] Waiting for connection context
0806 04:57:15.167208 (+ 74us) server_negotiation.cc:299] Negotiation successful
0806 04:57:15.167403 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7743,"thread_start_us":7572,"threads_started":1}
I20260806 04:57:15.168339 10831 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cc8e3bd8cf894e9c867f2050474a45c7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 04:57:15.171275 11033 tablet_service.cc:1467] Processing CreateTablet for tablet cc8e3bd8cf894e9c867f2050474a45c7 (DEFAULT_TABLE table=test-table [id=a6e349c196734706866389ce61577c22]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 04:57:15.172821 11033 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cc8e3bd8cf894e9c867f2050474a45c7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 04:57:15.177716 11150 tablet_bootstrap.cc:492] T cc8e3bd8cf894e9c867f2050474a45c7 P dcbd463d6deb462f960ac097fc3ebfea: Bootstrap starting.
I20260806 04:57:15.180912 11150 tablet_bootstrap.cc:654] T cc8e3bd8cf894e9c867f2050474a45c7 P dcbd463d6deb462f960ac097fc3ebfea: Neither blocks nor log segments found. Creating new log.
I20260806 04:57:15.181960 11150 log.cc:824] T cc8e3bd8cf894e9c867f2050474a45c7 P dcbd463d6deb462f960ac097fc3ebfea: Log is configured to *not* fsync() on all Append() calls
I20260806 04:57:15.186616 11150 tablet_bootstrap.cc:492] T cc8e3bd8cf894e9c867f2050474a45c7 P dcbd463d6deb462f960ac097fc3ebfea: No bootstrap required, opened a new log
I20260806 04:57:15.186820 11155 tablet_bootstrap.cc:492] T cc8e3bd8cf894e9c867f2050474a45c7 P 4922b622110d494389f3b220c71ca9e1: Bootstrap starting.
I20260806 04:57:15.186911 11150 ts_tablet_manager.cc:1397] T cc8e3bd8cf894e9c867f2050474a45c7 P dcbd463d6deb462f960ac097fc3ebfea: Time spent bootstrapping tablet: real 0.009s user 0.005s sys 0.004s
I20260806 04:57:15.188158 11158 tablet_bootstrap.cc:492] T cc8e3bd8cf894e9c867f2050474a45c7 P 71b0e7603bf149f0a526d89886013a38: Bootstrap starting.
I20260806 04:57:15.190299 11155 tablet_bootstrap.cc:654] T cc8e3bd8cf894e9c867f2050474a45c7 P 4922b622110d494389f3b220c71ca9e1: Neither blocks nor log segments found. Creating new log.
I20260806 04:57:15.191184 11158 tablet_bootstrap.cc:654] T cc8e3bd8cf894e9c867f2050474a45c7 P 71b0e7603bf149f0a526d89886013a38: Neither blocks nor log segments found. Creating new log.
I20260806 04:57:15.191576 11155 log.cc:824] T cc8e3bd8cf894e9c867f2050474a45c7 P 4922b622110d494389f3b220c71ca9e1: Log is configured to *not* fsync() on all Append() calls
I20260806 04:57:15.192525 11158 log.cc:824] T cc8e3bd8cf894e9c867f2050474a45c7 P 71b0e7603bf149f0a526d89886013a38: Log is configured to *not* fsync() on all Append() calls
I20260806 04:57:15.195691 11158 tablet_bootstrap.cc:492] T cc8e3bd8cf894e9c867f2050474a45c7 P 71b0e7603bf149f0a526d89886013a38: No bootstrap required, opened a new log
I20260806 04:57:15.195971 11158 ts_tablet_manager.cc:1397] T cc8e3bd8cf894e9c867f2050474a45c7 P 71b0e7603bf149f0a526d89886013a38: Time spent bootstrapping tablet: real 0.008s user 0.007s sys 0.000s
I20260806 04:57:15.196969 11150 raft_consensus.cc:348] T cc8e3bd8cf894e9c867f2050474a45c7 P dcbd463d6deb462f960ac097fc3ebfea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dcbd463d6deb462f960ac097fc3ebfea" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 46643 } } peers { permanent_uuid: "4922b622110d494389f3b220c71ca9e1" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 41241 } } peers { permanent_uuid: "71b0e7603bf149f0a526d89886013a38" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 45891 } }
I20260806 04:57:15.197302 11155 tablet_bootstrap.cc:492] T cc8e3bd8cf894e9c867f2050474a45c7 P 4922b622110d494389f3b220c71ca9e1: No bootstrap required, opened a new log
I20260806 04:57:15.197377 11150 raft_consensus.cc:374] T cc8e3bd8cf894e9c867f2050474a45c7 P dcbd463d6deb462f960ac097fc3ebfea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 04:57:15.197496 11150 raft_consensus.cc:729] T cc8e3bd8cf894e9c867f2050474a45c7 P dcbd463d6deb462f960ac097fc3ebfea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dcbd463d6deb462f960ac097fc3ebfea, State: Initialized, Role: FOLLOWER
I20260806 04:57:15.197619 11155 ts_tablet_manager.cc:1397] T cc8e3bd8cf894e9c867f2050474a45c7 P 4922b622110d494389f3b220c71ca9e1: Time spent bootstrapping tablet: real 0.013s user 0.005s sys 0.004s
I20260806 04:57:15.198128 11150 consensus_queue.cc:260] T cc8e3bd8cf894e9c867f2050474a45c7 P dcbd463d6deb462f960ac097fc3ebfea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dcbd463d6deb462f960ac097fc3ebfea" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 46643 } } peers { permanent_uuid: "4922b622110d494389f3b220c71ca9e1" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 41241 } } peers { permanent_uuid: "71b0e7603bf149f0a526d89886013a38" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 45891 } }
I20260806 04:57:15.200058 11150 ts_tablet_manager.cc:1428] T cc8e3bd8cf894e9c867f2050474a45c7 P dcbd463d6deb462f960ac097fc3ebfea: Time spent starting tablet: real 0.013s user 0.008s sys 0.004s
I20260806 04:57:15.208000 11158 raft_consensus.cc:348] T cc8e3bd8cf894e9c867f2050474a45c7 P 71b0e7603bf149f0a526d89886013a38 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dcbd463d6deb462f960ac097fc3ebfea" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 46643 } } peers { permanent_uuid: "4922b622110d494389f3b220c71ca9e1" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 41241 } } peers { permanent_uuid: "71b0e7603bf149f0a526d89886013a38" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 45891 } }
I20260806 04:57:15.208451 11158 raft_consensus.cc:374] T cc8e3bd8cf894e9c867f2050474a45c7 P 71b0e7603bf149f0a526d89886013a38 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 04:57:15.208590 11158 raft_consensus.cc:729] T cc8e3bd8cf894e9c867f2050474a45c7 P 71b0e7603bf149f0a526d89886013a38 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 71b0e7603bf149f0a526d89886013a38, State: Initialized, Role: FOLLOWER
I20260806 04:57:15.209060 11155 raft_consensus.cc:348] T cc8e3bd8cf894e9c867f2050474a45c7 P 4922b622110d494389f3b220c71ca9e1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dcbd463d6deb462f960ac097fc3ebfea" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 46643 } } peers { permanent_uuid: "4922b622110d494389f3b220c71ca9e1" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 41241 } } peers { permanent_uuid: "71b0e7603bf149f0a526d89886013a38" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 45891 } }
I20260806 04:57:15.209249 11158 consensus_queue.cc:260] T cc8e3bd8cf894e9c867f2050474a45c7 P 71b0e7603bf149f0a526d89886013a38 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dcbd463d6deb462f960ac097fc3ebfea" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 46643 } } peers { permanent_uuid: "4922b622110d494389f3b220c71ca9e1" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 41241 } } peers { permanent_uuid: "71b0e7603bf149f0a526d89886013a38" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 45891 } }
I20260806 04:57:15.209537 11155 raft_consensus.cc:374] T cc8e3bd8cf894e9c867f2050474a45c7 P 4922b622110d494389f3b220c71ca9e1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 04:57:15.209668 11155 raft_consensus.cc:729] T cc8e3bd8cf894e9c867f2050474a45c7 P 4922b622110d494389f3b220c71ca9e1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4922b622110d494389f3b220c71ca9e1, State: Initialized, Role: FOLLOWER
I20260806 04:57:15.210511 11126 heartbeater.cc:503] Master 127.9.96.126:33081 was elected leader, sending a full tablet report...
I20260806 04:57:15.210721 11155 consensus_queue.cc:260] T cc8e3bd8cf894e9c867f2050474a45c7 P 4922b622110d494389f3b220c71ca9e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dcbd463d6deb462f960ac097fc3ebfea" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 46643 } } peers { permanent_uuid: "4922b622110d494389f3b220c71ca9e1" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 41241 } } peers { permanent_uuid: "71b0e7603bf149f0a526d89886013a38" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 45891 } }
I20260806 04:57:15.211194 11158 ts_tablet_manager.cc:1428] T cc8e3bd8cf894e9c867f2050474a45c7 P 71b0e7603bf149f0a526d89886013a38: Time spent starting tablet: real 0.015s user 0.014s sys 0.000s
I20260806 04:57:15.211925 10930 heartbeater.cc:503] Master 127.9.96.126:33081 was elected leader, sending a full tablet report...
I20260806 04:57:15.212563 11155 ts_tablet_manager.cc:1428] T cc8e3bd8cf894e9c867f2050474a45c7 P 4922b622110d494389f3b220c71ca9e1: Time spent starting tablet: real 0.015s user 0.012s sys 0.001s
W20260806 04:57:15.314652 11128 tablet.cc:2369] T cc8e3bd8cf894e9c867f2050474a45c7 P 71b0e7603bf149f0a526d89886013a38: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 04:57:15.320394 10931 tablet.cc:2369] T cc8e3bd8cf894e9c867f2050474a45c7 P 4922b622110d494389f3b220c71ca9e1: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 04:57:15.329277 10729 tablet.cc:2369] T cc8e3bd8cf894e9c867f2050474a45c7 P dcbd463d6deb462f960ac097fc3ebfea: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 04:57:15.335843 11163 raft_consensus.cc:482] T cc8e3bd8cf894e9c867f2050474a45c7 P dcbd463d6deb462f960ac097fc3ebfea [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 04:57:15.336145 11163 raft_consensus.cc:504] T cc8e3bd8cf894e9c867f2050474a45c7 P dcbd463d6deb462f960ac097fc3ebfea [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dcbd463d6deb462f960ac097fc3ebfea" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 46643 } } peers { permanent_uuid: "4922b622110d494389f3b220c71ca9e1" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 41241 } } peers { permanent_uuid: "71b0e7603bf149f0a526d89886013a38" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 45891 } }
I20260806 04:57:15.338428 11163 leader_election.cc:288] T cc8e3bd8cf894e9c867f2050474a45c7 P dcbd463d6deb462f960ac097fc3ebfea [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4922b622110d494389f3b220c71ca9e1 (127.9.96.66:41241), 71b0e7603bf149f0a526d89886013a38 (127.9.96.67:45891)
I20260806 04:57:15.347945 11148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:57:15.338914 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.96.65:36873 (local address 127.9.96.67:45891)
0806 04:57:15.339158 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0806 04:57:15.339165 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 04:57:15.340172 (+ 1007us) server_negotiation.cc:407] Connection header received
0806 04:57:15.340341 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:57:15.340345 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:57:15.340426 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:57:15.340511 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 04:57:15.341338 (+ 827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:57:15.342475 (+ 1137us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:57:15.345534 (+ 3059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:57:15.346824 (+ 1290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:57:15.346941 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:57:15.347034 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0806 04:57:15.347551 (+ 517us) server_negotiation.cc:299] Negotiation successful
0806 04:57:15.347680 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260806 04:57:15.347945 11146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:57:15.339310 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.96.65:50595 (local address 127.9.96.66:41241)
0806 04:57:15.339556 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0806 04:57:15.339562 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 04:57:15.340008 (+ 446us) server_negotiation.cc:407] Connection header received
0806 04:57:15.340258 (+ 250us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:57:15.340264 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:57:15.340347 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:57:15.340438 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 04:57:15.341339 (+ 901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:57:15.342475 (+ 1136us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:57:15.345542 (+ 3067us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:57:15.346779 (+ 1237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:57:15.346931 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:57:15.347027 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0806 04:57:15.347551 (+ 524us) server_negotiation.cc:299] Negotiation successful
0806 04:57:15.347680 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":98}
I20260806 04:57:15.348239 11170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:57:15.339164 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.67:45891 (local address 127.9.96.65:36873)
0806 04:57:15.340081 (+ 917us) negotiation.cc:107] Waiting for socket to connect
0806 04:57:15.340097 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 04:57:15.340215 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:57:15.340529 (+ 314us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:57:15.340533 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:57:15.340550 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 04:57:15.341130 (+ 580us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:57:15.341149 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:57:15.342614 (+ 1465us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:57:15.342618 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:57:15.345356 (+ 2738us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:57:15.345379 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:57:15.347090 (+ 1711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:57:15.347094 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:57:15.347340 (+ 246us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:57:15.347413 (+ 73us) client_negotiation.cc:769] Sending connection context
0806 04:57:15.347508 (+ 95us) client_negotiation.cc:240] Negotiation successful
0806 04:57:15.347749 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":802,"mutex_wait_us":332,"thread_start_us":573,"threads_started":1}
I20260806 04:57:15.348263 11168 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:57:15.338847 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.66:41241 (local address 127.9.96.65:50595)
0806 04:57:15.339849 (+ 1002us) negotiation.cc:107] Waiting for socket to connect
0806 04:57:15.339884 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0806 04:57:15.340058 (+ 174us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:57:15.340497 (+ 439us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:57:15.340502 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:57:15.340530 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 04:57:15.341184 (+ 654us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:57:15.341199 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:57:15.342612 (+ 1413us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:57:15.342617 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:57:15.345365 (+ 2748us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:57:15.345378 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:57:15.347073 (+ 1695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:57:15.347081 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:57:15.347337 (+ 256us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:57:15.347412 (+ 75us) client_negotiation.cc:769] Sending connection context
0806 04:57:15.347508 (+ 96us) client_negotiation.cc:240] Negotiation successful
0806 04:57:15.347749 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":826,"mutex_wait_us":281,"thread_start_us":565,"threads_started":1}
I20260806 04:57:15.349560 10861 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cc8e3bd8cf894e9c867f2050474a45c7" candidate_uuid: "dcbd463d6deb462f960ac097fc3ebfea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4922b622110d494389f3b220c71ca9e1" is_pre_election: true
I20260806 04:57:15.349560 11059 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cc8e3bd8cf894e9c867f2050474a45c7" candidate_uuid: "dcbd463d6deb462f960ac097fc3ebfea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "71b0e7603bf149f0a526d89886013a38" is_pre_election: true
I20260806 04:57:15.350317 10861 raft_consensus.cc:2454] T cc8e3bd8cf894e9c867f2050474a45c7 P 4922b622110d494389f3b220c71ca9e1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dcbd463d6deb462f960ac097fc3ebfea in term 0.
I20260806 04:57:15.350929 10575 leader_election.cc:302] T cc8e3bd8cf894e9c867f2050474a45c7 P dcbd463d6deb462f960ac097fc3ebfea [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4922b622110d494389f3b220c71ca9e1, dcbd463d6deb462f960ac097fc3ebfea; no voters:
I20260806 04:57:15.351261 11163 raft_consensus.cc:2781] T cc8e3bd8cf894e9c867f2050474a45c7 P dcbd463d6deb462f960ac097fc3ebfea [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 04:57:15.351433 11163 raft_consensus.cc:482] T cc8e3bd8cf894e9c867f2050474a45c7 P dcbd463d6deb462f960ac097fc3ebfea [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 04:57:15.351569 11163 raft_consensus.cc:3037] T cc8e3bd8cf894e9c867f2050474a45c7 P dcbd463d6deb462f960ac097fc3ebfea [term 0 FOLLOWER]: Advancing to term 1
I20260806 04:57:15.352087 11059 raft_consensus.cc:2454] T cc8e3bd8cf894e9c867f2050474a45c7 P 71b0e7603bf149f0a526d89886013a38 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dcbd463d6deb462f960ac097fc3ebfea in term 0.
I20260806 04:57:15.353964 11163 raft_consensus.cc:504] T cc8e3bd8cf894e9c867f2050474a45c7 P dcbd463d6deb462f960ac097fc3ebfea [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dcbd463d6deb462f960ac097fc3ebfea" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 46643 } } peers { permanent_uuid: "4922b622110d494389f3b220c71ca9e1" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 41241 } } peers { permanent_uuid: "71b0e7603bf149f0a526d89886013a38" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 45891 } }
I20260806 04:57:15.354540 11163 leader_election.cc:288] T cc8e3bd8cf894e9c867f2050474a45c7 P dcbd463d6deb462f960ac097fc3ebfea [CANDIDATE]: Term 1 election: Requested vote from peers 4922b622110d494389f3b220c71ca9e1 (127.9.96.66:41241), 71b0e7603bf149f0a526d89886013a38 (127.9.96.67:45891)
I20260806 04:57:15.355022 10861 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cc8e3bd8cf894e9c867f2050474a45c7" candidate_uuid: "dcbd463d6deb462f960ac097fc3ebfea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4922b622110d494389f3b220c71ca9e1"
I20260806 04:57:15.355022 11059 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cc8e3bd8cf894e9c867f2050474a45c7" candidate_uuid: "dcbd463d6deb462f960ac097fc3ebfea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "71b0e7603bf149f0a526d89886013a38"
I20260806 04:57:15.355228 11059 raft_consensus.cc:3037] T cc8e3bd8cf894e9c867f2050474a45c7 P 71b0e7603bf149f0a526d89886013a38 [term 0 FOLLOWER]: Advancing to term 1
I20260806 04:57:15.355226 10861 raft_consensus.cc:3037] T cc8e3bd8cf894e9c867f2050474a45c7 P 4922b622110d494389f3b220c71ca9e1 [term 0 FOLLOWER]: Advancing to term 1
I20260806 04:57:15.358485 10861 raft_consensus.cc:2454] T cc8e3bd8cf894e9c867f2050474a45c7 P 4922b622110d494389f3b220c71ca9e1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dcbd463d6deb462f960ac097fc3ebfea in term 1.
I20260806 04:57:15.358498 11059 raft_consensus.cc:2454] T cc8e3bd8cf894e9c867f2050474a45c7 P 71b0e7603bf149f0a526d89886013a38 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dcbd463d6deb462f960ac097fc3ebfea in term 1.
I20260806 04:57:15.358897 10575 leader_election.cc:302] T cc8e3bd8cf894e9c867f2050474a45c7 P dcbd463d6deb462f960ac097fc3ebfea [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4922b622110d494389f3b220c71ca9e1, dcbd463d6deb462f960ac097fc3ebfea; no voters:
I20260806 04:57:15.359185 11163 raft_consensus.cc:2781] T cc8e3bd8cf894e9c867f2050474a45c7 P dcbd463d6deb462f960ac097fc3ebfea [term 1 FOLLOWER]: Leader election won for term 1
I20260806 04:57:15.359678 11163 raft_consensus.cc:686] T cc8e3bd8cf894e9c867f2050474a45c7 P dcbd463d6deb462f960ac097fc3ebfea [term 1 LEADER]: Becoming Leader. State: Replica: dcbd463d6deb462f960ac097fc3ebfea, State: Running, Role: LEADER
I20260806 04:57:15.360124 11163 consensus_queue.cc:237] T cc8e3bd8cf894e9c867f2050474a45c7 P dcbd463d6deb462f960ac097fc3ebfea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dcbd463d6deb462f960ac097fc3ebfea" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 46643 } } peers { permanent_uuid: "4922b622110d494389f3b220c71ca9e1" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 41241 } } peers { permanent_uuid: "71b0e7603bf149f0a526d89886013a38" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 45891 } }
I20260806 04:57:15.365352 10468 catalog_manager.cc:5495] T cc8e3bd8cf894e9c867f2050474a45c7 P dcbd463d6deb462f960ac097fc3ebfea reported cstate change: term changed from 0 to 1, leader changed from <none> to dcbd463d6deb462f960ac097fc3ebfea (127.9.96.65). New cstate: current_term: 1 leader_uuid: "dcbd463d6deb462f960ac097fc3ebfea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dcbd463d6deb462f960ac097fc3ebfea" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 46643 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4922b622110d494389f3b220c71ca9e1" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 41241 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "71b0e7603bf149f0a526d89886013a38" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 45891 } health_report { overall_health: UNKNOWN } } }
Aug 06 04:57:15 dist-test-slave-x4qh krb5kdc[10398](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992233, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.96.65@KRBTEST.COM
I20260806 04:57:15.406504 11145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:57:15.392003 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55870 (local address 127.9.96.65:46643)
0806 04:57:15.392200 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0806 04:57:15.392205 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 04:57:15.392747 (+ 542us) server_negotiation.cc:407] Connection header received
0806 04:57:15.392965 (+ 218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:57:15.392974 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:57:15.393047 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:57:15.393156 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:57:15.394724 (+ 1568us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:57:15.395580 (+ 856us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:57:15.396866 (+ 1286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:57:15.397286 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:57:15.397445 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:57:15.401377 (+ 3932us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:57:15.401391 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:57:15.401407 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:57:15.401441 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:57:15.404283 (+ 2842us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:57:15.404872 (+ 589us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:57:15.404879 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:57:15.404881 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:57:15.405035 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:57:15.405343 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:57:15.405346 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:57:15.405348 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:57:15.405652 (+ 304us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:57:15.405874 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0806 04:57:15.406108 (+ 234us) server_negotiation.cc:299] Negotiation successful
0806 04:57:15.406270 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
Aug 06 04:57:15 dist-test-slave-x4qh krb5kdc[10398](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992235, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 06 04:57:15 dist-test-slave-x4qh krb5kdc[10398](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992235, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.9.96.65@KRBTEST.COM
I20260806 04:57:15.456457 11145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:57:15.442241 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55872 (local address 127.9.96.65:46643)
0806 04:57:15.442453 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0806 04:57:15.442462 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 04:57:15.442869 (+ 407us) server_negotiation.cc:407] Connection header received
0806 04:57:15.443105 (+ 236us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:57:15.443110 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:57:15.443186 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:57:15.443302 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:57:15.444793 (+ 1491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:57:15.445575 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:57:15.446946 (+ 1371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:57:15.447466 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:57:15.447610 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:57:15.451585 (+ 3975us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:57:15.451590 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:57:15.451593 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:57:15.451630 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:57:15.454242 (+ 2612us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:57:15.454837 (+ 595us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:57:15.454841 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:57:15.454843 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:57:15.454966 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:57:15.455318 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:57:15.455321 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:57:15.455323 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:57:15.455629 (+ 306us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:57:15.455866 (+ 237us) server_negotiation.cc:1035] Waiting for connection context
0806 04:57:15.456035 (+ 169us) server_negotiation.cc:299] Negotiation successful
0806 04:57:15.456214 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
Aug 06 04:57:15 dist-test-slave-x4qh krb5kdc[10398](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992235, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 06 04:57:15 dist-test-slave-x4qh krb5kdc[10398](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992235, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.96.65@KRBTEST.COM
I20260806 04:57:15.509804 11145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:57:15.489333 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55888 (local address 127.9.96.65:46643)
0806 04:57:15.489550 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0806 04:57:15.489559 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 04:57:15.489900 (+ 341us) server_negotiation.cc:407] Connection header received
0806 04:57:15.490098 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:57:15.490111 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:57:15.490185 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:57:15.490293 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:57:15.491721 (+ 1428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:57:15.492888 (+ 1167us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:57:15.494077 (+ 1189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:57:15.494803 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:57:15.494947 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:57:15.502933 (+ 7986us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:57:15.502939 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:57:15.502942 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:57:15.502976 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:57:15.506133 (+ 3157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:57:15.507014 (+ 881us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:57:15.507020 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:57:15.507024 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:57:15.507236 (+ 212us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:57:15.507779 (+ 543us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:57:15.507784 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:57:15.507788 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:57:15.508295 (+ 507us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:57:15.508662 (+ 367us) server_negotiation.cc:1035] Waiting for connection context
0806 04:57:15.509013 (+ 351us) server_negotiation.cc:299] Negotiation successful
0806 04:57:15.509286 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
W20260806 04:57:15.512792 10609 tablet_service.cc:2908] Rejecting scan request for tablet cc8e3bd8cf894e9c867f2050474a45c7: Uninitialized: clean time has not yet been initialized
I20260806 04:57:15.520080 9601 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu pid 10552
I20260806 04:57:15.538677 11145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:57:15.521866 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55894 (local address 127.9.96.65:46643)
0806 04:57:15.522107 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0806 04:57:15.522116 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 04:57:15.522144 (+ 28us) server_negotiation.cc:407] Connection header received
0806 04:57:15.522221 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:57:15.522230 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:57:15.522373 (+ 143us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:57:15.522538 (+ 165us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:57:15.524271 (+ 1733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:57:15.525351 (+ 1080us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:57:15.527507 (+ 2156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:57:15.528327 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:57:15.528471 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:57:15.531593 (+ 3122us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:57:15.531603 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:57:15.531612 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:57:15.531662 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:57:15.535662 (+ 4000us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:57:15.536539 (+ 877us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:57:15.536543 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:57:15.536545 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:57:15.536666 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:57:15.537073 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:57:15.537076 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:57:15.537078 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:57:15.537372 (+ 294us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:57:15.537625 (+ 253us) server_negotiation.cc:1035] Waiting for connection context
0806 04:57:15.537996 (+ 371us) server_negotiation.cc:299] Negotiation successful
0806 04:57:15.538243 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260806 04:57:15.539606 10723 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 04:57:15.640275 9601 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu with pid 10552
I20260806 04:57:15.664265 9601 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu pid 10735
Aug 06 04:57:15 dist-test-slave-x4qh krb5kdc[10398](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992235, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.96.66@KRBTEST.COM
I20260806 04:57:15.678486 11146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:57:15.665158 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38594 (local address 127.9.96.66:41241)
0806 04:57:15.665342 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0806 04:57:15.665346 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 04:57:15.665365 (+ 19us) server_negotiation.cc:407] Connection header received
0806 04:57:15.665427 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:57:15.665431 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:57:15.665518 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:57:15.665605 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:57:15.666827 (+ 1222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:57:15.667713 (+ 886us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:57:15.668948 (+ 1235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:57:15.669323 (+ 375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:57:15.669411 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:57:15.673276 (+ 3865us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:57:15.673288 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:57:15.673304 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:57:15.673336 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:57:15.676293 (+ 2957us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:57:15.676903 (+ 610us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:57:15.676910 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:57:15.676912 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:57:15.677028 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:57:15.677337 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:57:15.677340 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:57:15.677342 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:57:15.677685 (+ 343us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:57:15.677931 (+ 246us) server_negotiation.cc:1035] Waiting for connection context
0806 04:57:15.678093 (+ 162us) server_negotiation.cc:299] Negotiation successful
0806 04:57:15.678242 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260806 04:57:15.679219 10921 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 04:57:15.767284 9601 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu with pid 10735
I20260806 04:57:15.792766 9601 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu pid 10940
Aug 06 04:57:15 dist-test-slave-x4qh krb5kdc[10398](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992235, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.96.67@KRBTEST.COM
I20260806 04:57:15.807760 11148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:57:15.793641 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40112 (local address 127.9.96.67:45891)
0806 04:57:15.793854 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0806 04:57:15.793859 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 04:57:15.793906 (+ 47us) server_negotiation.cc:407] Connection header received
0806 04:57:15.793969 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:57:15.793973 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:57:15.794076 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:57:15.794168 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:57:15.795360 (+ 1192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:57:15.796207 (+ 847us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:57:15.797549 (+ 1342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:57:15.797996 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:57:15.798096 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:57:15.802115 (+ 4019us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:57:15.802126 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:57:15.802142 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:57:15.802173 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:57:15.804953 (+ 2780us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:57:15.805620 (+ 667us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:57:15.805627 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:57:15.805630 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:57:15.805768 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:57:15.806149 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:57:15.806154 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:57:15.806159 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:57:15.806511 (+ 352us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:57:15.806841 (+ 330us) server_negotiation.cc:1035] Waiting for connection context
0806 04:57:15.807139 (+ 298us) server_negotiation.cc:299] Negotiation successful
0806 04:57:15.807422 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260806 04:57:15.808382 11117 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 04:57:15.894289 9601 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu with pid 10940
I20260806 04:57:15.918819 9601 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu pid 10417
I20260806 04:57:15.920039 10509 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 04:57:16.025377 9601 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu with pid 10417
2026-08-06T04:57:16Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (4396 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785992227050689-9601-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 04:57:16 dist-test-slave-x4qh krb5kdc[11195](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 04:57:16 dist-test-slave-x4qh krb5kdc[11195](info): set up 2 sockets
Aug 06 04:57:16 dist-test-slave-x4qh krb5kdc[11195](info): commencing operation
krb5kdc: starting...
W20260806 04:57:17.299541 9601 mini_kdc.cc:121] Time spent starting KDC: real 1.223s user 0.000s sys 0.019s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785992227050689-9601-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785992227050689-9601-0/krb5kdc/test-user.keytab.
Aug 06 04:57:17 dist-test-slave-x4qh krb5kdc[11195](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992237, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T04:57:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T04:57:17Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.96.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.96.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.96.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.126.keytab.
Entry for principal kudu/127.9.96.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.126.keytab.
I20260806 04:57:17.486985 9601 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
/tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785992227050689-9601-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785992227050689-9601-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785992227050689-9601-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785992227050689-9601-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.96.126:34519
--webserver_interface=127.9.96.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.126.keytab
--principal=kudu/127.9.96.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.96.84:41399
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.96.126:34519
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785992227050689-9601-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785992227050689-9601-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785992227050689-9601-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 04:57:17.810624 11215 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 04:57:17.810918 11215 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 04:57:17.811142 11215 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 04:57:17.818399 11215 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 04:57:17.818502 11215 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260806 04:57:17.818617 11215 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 04:57:17.818743 11215 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 04:57:17.818841 11215 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 04:57:17.818939 11215 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 04:57:17.828785 11215 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.96.84:41399
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785992227050689-9601-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785992227050689-9601-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.96.126:34519
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.126.keytab
--principal=kudu/127.9.96.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.96.126:34519
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785992227050689-9601-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.96.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785992227050689-9601-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 03:56:12 UTC on ccycloud.quasar-guijbm.root.comops.site
build id 4569618
ASAN enabled
I20260806 04:57:17.830029 11215 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 04:57:17.831429 11215 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 04:57:17.843742 11220 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:57:17.843842 11223 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:57:17.843796 11225 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:57:17.845079 11215 server_base.cc:1034] running on GCE node
I20260806 04:57:17.845924 11215 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 04:57:17.847275 11215 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 04:57:17.848555 11215 hybrid_clock.cc:648] HybridClock initialized: now 1785992237848476 us; error 44 us; skew 500 ppm
I20260806 04:57:17.848932 11215 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 04:57:17 dist-test-slave-x4qh krb5kdc[11195](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992237, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.96.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 04:57:17.852275 11215 init.cc:377] Logged in from keytab as kudu/127.9.96.126@KRBTEST.COM (short username kudu)
I20260806 04:57:17.853803 11215 webserver.cc:460] Webserver started at http://127.9.96.126:45549/ using document root <none> and password file <none>
I20260806 04:57:17.854537 11215 fs_manager.cc:359] Metadata directory not provided
I20260806 04:57:17.854694 11215 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 04:57:17.855042 11215 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 04:57:17.858090 11215 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785992227050689-9601-0/minicluster-data/master-0/data/instance:
uuid: "d1b98044e21d4333a884f6762082b055"
format_stamp: "Formatted at 2026-08-06 04:57:17 on dist-test-slave-x4qh"
server_key: "5aff68983fc37ce6e824bc6a0197fd46"
server_key_iv: "77bc46383ff15624c8de5411eb1ace7f"
server_key_version: "encryptionkey@0"
I20260806 04:57:17.859014 11215 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785992227050689-9601-0/minicluster-data/master-0/wal/instance:
uuid: "d1b98044e21d4333a884f6762082b055"
format_stamp: "Formatted at 2026-08-06 04:57:17 on dist-test-slave-x4qh"
server_key: "5aff68983fc37ce6e824bc6a0197fd46"
server_key_iv: "77bc46383ff15624c8de5411eb1ace7f"
server_key_version: "encryptionkey@0"
I20260806 04:57:17.864427 11215 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260806 04:57:17.867903 11238 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 04:57:17.869156 11215 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260806 04:57:17.869305 11215 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785992227050689-9601-0/minicluster-data/master-0/data,/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785992227050689-9601-0/minicluster-data/master-0/wal
uuid: "d1b98044e21d4333a884f6762082b055"
format_stamp: "Formatted at 2026-08-06 04:57:17 on dist-test-slave-x4qh"
server_key: "5aff68983fc37ce6e824bc6a0197fd46"
server_key_iv: "77bc46383ff15624c8de5411eb1ace7f"
server_key_version: "encryptionkey@0"
I20260806 04:57:17.869477 11215 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785992227050689-9601-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785992227050689-9601-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785992227050689-9601-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 04:57:17.880910 11215 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 04:57:17.885108 11215 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 04:57:17.885327 11215 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 04:57:17.907207 11215 rpc_server.cc:307] RPC server started. Bound to: 127.9.96.126:34519
I20260806 04:57:17.907226 11333 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.96.126:34519 every 8 connection(s)
I20260806 04:57:17.908682 11215 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785992227050689-9601-0/minicluster-data/master-0/data/info.pb
I20260806 04:57:17.912241 9601 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu as pid 11215
I20260806 04:57:17.912477 9601 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785992227050689-9601-0/minicluster-data/master-0/wal/instance
I20260806 04:57:17.912618 11335 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 04:57:17.913309 9601 external_mini_cluster.cc:1428] Setting key 70d542b215e956ccc20e96402bbdd76c
I20260806 04:57:17.926697 11335 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d1b98044e21d4333a884f6762082b055: Bootstrap starting.
Aug 06 04:57:17 dist-test-slave-x4qh krb5kdc[11195](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992237, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.96.126@KRBTEST.COM
I20260806 04:57:17.929817 11335 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d1b98044e21d4333a884f6762082b055: Neither blocks nor log segments found. Creating new log.
I20260806 04:57:17.931165 11335 log.cc:824] T 00000000000000000000000000000000 P d1b98044e21d4333a884f6762082b055: Log is configured to *not* fsync() on all Append() calls
I20260806 04:57:17.934530 11335 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d1b98044e21d4333a884f6762082b055: No bootstrap required, opened a new log
I20260806 04:57:17.938211 11339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:57:17.915402 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51902 (local address 127.9.96.126:34519)
0806 04:57:17.916319 (+ 917us) server_negotiation.cc:206] Beginning negotiation
0806 04:57:17.916333 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0806 04:57:17.916398 (+ 65us) server_negotiation.cc:407] Connection header received
0806 04:57:17.918267 (+ 1869us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:57:17.918298 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:57:17.918758 (+ 460us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:57:17.919183 (+ 425us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:57:17.920752 (+ 1569us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:57:17.922716 (+ 1964us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:57:17.924129 (+ 1413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:57:17.925213 (+ 1084us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:57:17.925397 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:57:17.929725 (+ 4328us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:57:17.929737 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:57:17.929755 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:57:17.929803 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:57:17.933701 (+ 3898us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:57:17.934408 (+ 707us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:57:17.934416 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:57:17.934424 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:57:17.934667 (+ 243us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:57:17.935076 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:57:17.935080 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:57:17.935083 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:57:17.935804 (+ 721us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:57:17.936106 (+ 302us) server_negotiation.cc:1035] Waiting for connection context
0806 04:57:17.936464 (+ 358us) server_negotiation.cc:299] Negotiation successful
0806 04:57:17.936894 (+ 430us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":661,"thread_start_us":430,"threads_started":1}
I20260806 04:57:17.943758 11335 raft_consensus.cc:348] T 00000000000000000000000000000000 P d1b98044e21d4333a884f6762082b055 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d1b98044e21d4333a884f6762082b055" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 34519 } }
I20260806 04:57:17.944065 11335 raft_consensus.cc:374] T 00000000000000000000000000000000 P d1b98044e21d4333a884f6762082b055 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 04:57:17.944182 11335 raft_consensus.cc:729] T 00000000000000000000000000000000 P d1b98044e21d4333a884f6762082b055 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d1b98044e21d4333a884f6762082b055, State: Initialized, Role: FOLLOWER
I20260806 04:57:17.944736 11335 consensus_queue.cc:260] T 00000000000000000000000000000000 P d1b98044e21d4333a884f6762082b055 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d1b98044e21d4333a884f6762082b055" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 34519 } }
I20260806 04:57:17.944949 11335 raft_consensus.cc:388] T 00000000000000000000000000000000 P d1b98044e21d4333a884f6762082b055 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 04:57:17.945042 11335 raft_consensus.cc:482] T 00000000000000000000000000000000 P d1b98044e21d4333a884f6762082b055 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 04:57:17.945163 11335 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d1b98044e21d4333a884f6762082b055 [term 0 FOLLOWER]: Advancing to term 1
I20260806 04:57:17.947537 11335 raft_consensus.cc:504] T 00000000000000000000000000000000 P d1b98044e21d4333a884f6762082b055 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d1b98044e21d4333a884f6762082b055" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 34519 } }
I20260806 04:57:17.947978 11335 leader_election.cc:302] T 00000000000000000000000000000000 P d1b98044e21d4333a884f6762082b055 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d1b98044e21d4333a884f6762082b055; no voters:
I20260806 04:57:17.948470 11335 leader_election.cc:288] T 00000000000000000000000000000000 P d1b98044e21d4333a884f6762082b055 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 04:57:17.948572 11343 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d1b98044e21d4333a884f6762082b055 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 04:57:17.949366 11343 raft_consensus.cc:686] T 00000000000000000000000000000000 P d1b98044e21d4333a884f6762082b055 [term 1 LEADER]: Becoming Leader. State: Replica: d1b98044e21d4333a884f6762082b055, State: Running, Role: LEADER
I20260806 04:57:17.949981 11343 consensus_queue.cc:237] T 00000000000000000000000000000000 P d1b98044e21d4333a884f6762082b055 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d1b98044e21d4333a884f6762082b055" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 34519 } }
I20260806 04:57:17.950666 11335 sys_catalog.cc:564] T 00000000000000000000000000000000 P d1b98044e21d4333a884f6762082b055 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 04:57:17.952569 11344 sys_catalog.cc:455] T 00000000000000000000000000000000 P d1b98044e21d4333a884f6762082b055 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d1b98044e21d4333a884f6762082b055" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d1b98044e21d4333a884f6762082b055" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 34519 } } }
I20260806 04:57:17.952936 11344 sys_catalog.cc:458] T 00000000000000000000000000000000 P d1b98044e21d4333a884f6762082b055 [sys.catalog]: This master's current role is: LEADER
I20260806 04:57:17.953639 11345 sys_catalog.cc:455] T 00000000000000000000000000000000 P d1b98044e21d4333a884f6762082b055 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d1b98044e21d4333a884f6762082b055. Latest consensus state: current_term: 1 leader_uuid: "d1b98044e21d4333a884f6762082b055" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d1b98044e21d4333a884f6762082b055" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 34519 } } }
I20260806 04:57:17.954002 11345 sys_catalog.cc:458] T 00000000000000000000000000000000 P d1b98044e21d4333a884f6762082b055 [sys.catalog]: This master's current role is: LEADER
I20260806 04:57:17.954836 11350 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 04:57:17.960371 11350 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 04:57:17.968715 11350 catalog_manager.cc:1347] Generated new cluster ID: 87544d46db3e45d184e170724e2a3090
I20260806 04:57:17.968830 11350 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 04:57:17.989795 11350 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 04:57:17.991034 11350 catalog_manager.cc:1504] Loading token signing keys...
I20260806 04:57:17.999621 11350 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d1b98044e21d4333a884f6762082b055: Generated new TSK 0
I20260806 04:57:18.000486 11350 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.9.96.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.96.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.96.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.65.keytab.
Entry for principal kudu/127.9.96.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.65.keytab.
I20260806 04:57:18.064973 9601 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
/tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785992227050689-9601-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785992227050689-9601-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785992227050689-9601-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785992227050689-9601-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.96.65:0
--local_ip_for_outbound_sockets=127.9.96.65
--webserver_interface=127.9.96.65
--webserver_port=0
--tserver_master_addrs=127.9.96.126:34519
--keytab_file=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.65.keytab
--principal=kudu/127.9.96.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.96.84:41399
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785992227050689-9601-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785992227050689-9601-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785992227050689-9601-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 04:57:18.392719 11373 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 04:57:18.392969 11373 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 04:57:18.393061 11373 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 04:57:18.399982 11373 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 04:57:18.400085 11373 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 04:57:18.400167 11373 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 04:57:18.400249 11373 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.96.65
I20260806 04:57:18.409530 11373 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.96.84:41399
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785992227050689-9601-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785992227050689-9601-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.65.keytab
--principal=kudu/127.9.96.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.96.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785992227050689-9601-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.96.65
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.9.96.126:34519
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.96.65
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785992227050689-9601-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 03:56:12 UTC on ccycloud.quasar-guijbm.root.comops.site
build id 4569618
ASAN enabled
I20260806 04:57:18.410698 11373 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 04:57:18.411986 11373 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 04:57:18.424943 11382 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:57:18.424906 11380 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:57:18.424989 11379 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:57:18.426108 11373 server_base.cc:1034] running on GCE node
I20260806 04:57:18.426697 11373 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 04:57:18.427665 11373 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 04:57:18.428912 11373 hybrid_clock.cc:648] HybridClock initialized: now 1785992238428853 us; error 30 us; skew 500 ppm
I20260806 04:57:18.429211 11373 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 04:57:18 dist-test-slave-x4qh krb5kdc[11195](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992238, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.96.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 04:57:18.432541 11373 init.cc:377] Logged in from keytab as kudu/127.9.96.65@KRBTEST.COM (short username kudu)
I20260806 04:57:18.433923 11373 webserver.cc:460] Webserver started at http://127.9.96.65:36581/ using document root <none> and password file <none>
I20260806 04:57:18.434604 11373 fs_manager.cc:359] Metadata directory not provided
I20260806 04:57:18.434743 11373 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 04:57:18.435084 11373 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 04:57:18.438014 11373 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785992227050689-9601-0/minicluster-data/ts-0/data/instance:
uuid: "bb34b1fe2b6a439fb22a40a0e22d9607"
format_stamp: "Formatted at 2026-08-06 04:57:18 on dist-test-slave-x4qh"
server_key: "3abadf19f368cc00673871ae6e5b2ab4"
server_key_iv: "94fb850b847b20652dab2e1e6732d486"
server_key_version: "encryptionkey@0"
I20260806 04:57:18.438922 11373 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785992227050689-9601-0/minicluster-data/ts-0/wal/instance:
uuid: "bb34b1fe2b6a439fb22a40a0e22d9607"
format_stamp: "Formatted at 2026-08-06 04:57:18 on dist-test-slave-x4qh"
server_key: "3abadf19f368cc00673871ae6e5b2ab4"
server_key_iv: "94fb850b847b20652dab2e1e6732d486"
server_key_version: "encryptionkey@0"
I20260806 04:57:18.444175 11373 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260806 04:57:18.447652 11392 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 04:57:18.448917 11373 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260806 04:57:18.449108 11373 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785992227050689-9601-0/minicluster-data/ts-0/data,/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785992227050689-9601-0/minicluster-data/ts-0/wal
uuid: "bb34b1fe2b6a439fb22a40a0e22d9607"
format_stamp: "Formatted at 2026-08-06 04:57:18 on dist-test-slave-x4qh"
server_key: "3abadf19f368cc00673871ae6e5b2ab4"
server_key_iv: "94fb850b847b20652dab2e1e6732d486"
server_key_version: "encryptionkey@0"
I20260806 04:57:18.449324 11373 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785992227050689-9601-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785992227050689-9601-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785992227050689-9601-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 04:57:18.463690 11373 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 04:57:18.467634 11373 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 04:57:18.467911 11373 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 04:57:18.470590 11373 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 04:57:18.470697 11373 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 04:57:18.470857 11373 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 04:57:18.470943 11373 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 04:57:18 dist-test-slave-x4qh krb5kdc[11195](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992238, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.96.65@KRBTEST.COM for kudu/127.9.96.126@KRBTEST.COM
I20260806 04:57:18.494294 11412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:57:18.471479 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.96.65:37637 (local address 127.9.96.126:34519)
0806 04:57:18.472768 (+ 1289us) server_negotiation.cc:206] Beginning negotiation
0806 04:57:18.472774 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 04:57:18.473131 (+ 357us) server_negotiation.cc:407] Connection header received
0806 04:57:18.475314 (+ 2183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:57:18.475320 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:57:18.475441 (+ 121us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:57:18.475564 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:57:18.477684 (+ 2120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:57:18.479016 (+ 1332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:57:18.481694 (+ 2678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:57:18.482316 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:57:18.482461 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:57:18.487150 (+ 4689us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:57:18.487155 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:57:18.487158 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:57:18.487189 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:57:18.490421 (+ 3232us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:57:18.491502 (+ 1081us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:57:18.491507 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:57:18.491509 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:57:18.491626 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:57:18.492227 (+ 601us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:57:18.492231 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:57:18.492233 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:57:18.492614 (+ 381us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:57:18.492775 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0806 04:57:18.493697 (+ 922us) server_negotiation.cc:299] Negotiation successful
0806 04:57:18.493912 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1141,"thread_start_us":1037,"threads_started":1}
I20260806 04:57:18.495687 11413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:57:18.472006 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:34519 (local address 127.9.96.65:37637)
0806 04:57:18.472963 (+ 957us) negotiation.cc:107] Waiting for socket to connect
0806 04:57:18.473016 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0806 04:57:18.475034 (+ 2018us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:57:18.475804 (+ 770us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:57:18.475812 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:57:18.476586 (+ 774us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:57:18.477465 (+ 879us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:57:18.477498 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:57:18.479212 (+ 1714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:57:18.479220 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:57:18.481475 (+ 2255us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:57:18.481492 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:57:18.482554 (+ 1062us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:57:18.482565 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:57:18.482801 (+ 236us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:57:18.483735 (+ 934us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:57:18.483771 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:57:18.486912 (+ 3141us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 04:57:18.490583 (+ 3671us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:57:18.490597 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:57:18.490616 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:57:18.491341 (+ 725us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:57:18.491758 (+ 417us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:57:18.491763 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:57:18.491767 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:57:18.492066 (+ 299us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:57:18.492847 (+ 781us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 04:57:18.492852 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 04:57:18.493279 (+ 427us) client_negotiation.cc:769] Sending connection context
0806 04:57:18.493606 (+ 327us) client_negotiation.cc:240] Negotiation successful
0806 04:57:18.494125 (+ 519us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":684,"thread_start_us":437,"threads_started":1}
I20260806 04:57:18.518569 11412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:57:18.503539 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.96.65:38303 (local address 127.9.96.126:34519)
0806 04:57:18.503736 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0806 04:57:18.503745 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 04:57:18.504236 (+ 491us) server_negotiation.cc:407] Connection header received
0806 04:57:18.504349 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:57:18.504356 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:57:18.504491 (+ 135us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:57:18.504621 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:57:18.505454 (+ 833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:57:18.506833 (+ 1379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:57:18.508689 (+ 1856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:57:18.509412 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:57:18.509558 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:57:18.512185 (+ 2627us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:57:18.512194 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:57:18.512199 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:57:18.512240 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:57:18.515975 (+ 3735us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:57:18.516622 (+ 647us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:57:18.516628 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:57:18.516631 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:57:18.516768 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:57:18.517147 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:57:18.517152 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:57:18.517157 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:57:18.517415 (+ 258us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:57:18.517593 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0806 04:57:18.517986 (+ 393us) server_negotiation.cc:299] Negotiation successful
0806 04:57:18.518219 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260806 04:57:18.518666 11531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:57:18.503300 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:34519 (local address 127.9.96.65:38303)
0806 04:57:18.504126 (+ 826us) negotiation.cc:107] Waiting for socket to connect
0806 04:57:18.504149 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0806 04:57:18.504254 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:57:18.504634 (+ 380us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:57:18.504637 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:57:18.505027 (+ 390us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:57:18.505314 (+ 287us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:57:18.505322 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:57:18.506992 (+ 1670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:57:18.506996 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:57:18.508419 (+ 1423us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:57:18.508432 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:57:18.509606 (+ 1174us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:57:18.509611 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:57:18.509758 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:57:18.510399 (+ 641us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:57:18.510435 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:57:18.511923 (+ 1488us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 04:57:18.516099 (+ 4176us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:57:18.516103 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:57:18.516105 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:57:18.516484 (+ 379us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:57:18.516888 (+ 404us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:57:18.516899 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:57:18.516907 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:57:18.517027 (+ 120us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:57:18.517551 (+ 524us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 04:57:18.517555 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 04:57:18.517737 (+ 182us) client_negotiation.cc:769] Sending connection context
0806 04:57:18.517873 (+ 136us) client_negotiation.cc:240] Negotiation successful
0806 04:57:18.518212 (+ 339us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":683,"thread_start_us":541,"threads_started":1}
I20260806 04:57:18.522586 11373 rpc_server.cc:307] RPC server started. Bound to: 127.9.96.65:39511
I20260806 04:57:18.522604 11574 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.96.65:39511 every 8 connection(s)
I20260806 04:57:18.523967 11373 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785992227050689-9601-0/minicluster-data/ts-0/data/info.pb
I20260806 04:57:18.524784 9601 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu as pid 11373
I20260806 04:57:18.525022 9601 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785992227050689-9601-0/minicluster-data/ts-0/wal/instance
I20260806 04:57:18.525853 9601 external_mini_cluster.cc:1428] Setting key 1090f533d942e62a4d125b844471009e
I20260806 04:57:18.526168 11575 heartbeater.cc:347] Connected to a master server at 127.9.96.126:34519
I20260806 04:57:18.526402 11575 heartbeater.cc:464] Registering TS with master...
I20260806 04:57:18.527132 11575 heartbeater.cc:511] Master 127.9.96.126:34519 requested a full tablet report, sending...
I20260806 04:57:18.529295 11264 ts_manager.cc:194] Registered new tserver with Master: bb34b1fe2b6a439fb22a40a0e22d9607 (127.9.96.65:39511)
I20260806 04:57:18.531461 11264 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.96.65@KRBTEST.COM'} at 127.9.96.65:37637
WARNING: no policy specified for kudu/127.9.96.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.96.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.96.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.66.keytab.
Entry for principal kudu/127.9.96.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.66.keytab.
I20260806 04:57:18.568881 9601 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
/tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785992227050689-9601-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785992227050689-9601-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785992227050689-9601-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785992227050689-9601-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.96.66:0
--local_ip_for_outbound_sockets=127.9.96.66
--webserver_interface=127.9.96.66
--webserver_port=0
--tserver_master_addrs=127.9.96.126:34519
--keytab_file=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.66.keytab
--principal=kudu/127.9.96.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.96.84:41399
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785992227050689-9601-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785992227050689-9601-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785992227050689-9601-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 04:57:18.890767 11582 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 04:57:18.891024 11582 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 04:57:18.891120 11582 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 04:57:18.898319 11582 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 04:57:18.898468 11582 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 04:57:18.898607 11582 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 04:57:18.898744 11582 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.96.66
I20260806 04:57:18.909387 11582 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.96.84:41399
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785992227050689-9601-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785992227050689-9601-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.66.keytab
--principal=kudu/127.9.96.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.96.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785992227050689-9601-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.96.66
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.9.96.126:34519
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.96.66
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785992227050689-9601-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 03:56:12 UTC on ccycloud.quasar-guijbm.root.comops.site
build id 4569618
ASAN enabled
I20260806 04:57:18.910817 11582 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 04:57:18.912407 11582 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 04:57:18.926640 11587 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:57:18.926648 11588 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:57:18.927893 11591 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:57:18.928119 11582 server_base.cc:1034] running on GCE node
I20260806 04:57:18.928800 11582 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 04:57:18.929672 11582 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 04:57:18.930962 11582 hybrid_clock.cc:648] HybridClock initialized: now 1785992238930894 us; error 39 us; skew 500 ppm
I20260806 04:57:18.931295 11582 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 04:57:18 dist-test-slave-x4qh krb5kdc[11195](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992238, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.96.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 04:57:18.934603 11582 init.cc:377] Logged in from keytab as kudu/127.9.96.66@KRBTEST.COM (short username kudu)
I20260806 04:57:18.936077 11582 webserver.cc:460] Webserver started at http://127.9.96.66:44815/ using document root <none> and password file <none>
I20260806 04:57:18.936825 11582 fs_manager.cc:359] Metadata directory not provided
I20260806 04:57:18.936923 11582 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 04:57:18.937207 11582 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 04:57:18.940097 11582 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785992227050689-9601-0/minicluster-data/ts-1/data/instance:
uuid: "3c92b56d4ab3443b9c85bdabf4a72977"
format_stamp: "Formatted at 2026-08-06 04:57:18 on dist-test-slave-x4qh"
server_key: "495c914d2b80e5973e4daaa75d0953a4"
server_key_iv: "6a58ebc5f4e1368103a34ef721ca3988"
server_key_version: "encryptionkey@0"
I20260806 04:57:18.940869 11582 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785992227050689-9601-0/minicluster-data/ts-1/wal/instance:
uuid: "3c92b56d4ab3443b9c85bdabf4a72977"
format_stamp: "Formatted at 2026-08-06 04:57:18 on dist-test-slave-x4qh"
server_key: "495c914d2b80e5973e4daaa75d0953a4"
server_key_iv: "6a58ebc5f4e1368103a34ef721ca3988"
server_key_version: "encryptionkey@0"
I20260806 04:57:18.945874 11582 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260806 04:57:18.949277 11604 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 04:57:18.950577 11582 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 04:57:18.950698 11582 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785992227050689-9601-0/minicluster-data/ts-1/data,/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785992227050689-9601-0/minicluster-data/ts-1/wal
uuid: "3c92b56d4ab3443b9c85bdabf4a72977"
format_stamp: "Formatted at 2026-08-06 04:57:18 on dist-test-slave-x4qh"
server_key: "495c914d2b80e5973e4daaa75d0953a4"
server_key_iv: "6a58ebc5f4e1368103a34ef721ca3988"
server_key_version: "encryptionkey@0"
I20260806 04:57:18.950868 11582 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785992227050689-9601-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785992227050689-9601-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785992227050689-9601-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 04:57:18.964819 11582 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 04:57:18.969103 11582 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 04:57:18.969331 11582 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 04:57:18.971777 11582 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 04:57:18.971879 11582 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 04:57:18.972021 11582 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 04:57:18.972124 11582 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 04:57:18 dist-test-slave-x4qh krb5kdc[11195](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992238, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.96.66@KRBTEST.COM for kudu/127.9.96.126@KRBTEST.COM
I20260806 04:57:18.996203 11412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:57:18.972403 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.96.66:46585 (local address 127.9.96.126:34519)
0806 04:57:18.972594 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0806 04:57:18.972601 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 04:57:18.974040 (+ 1439us) server_negotiation.cc:407] Connection header received
0806 04:57:18.976161 (+ 2121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:57:18.976167 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:57:18.976308 (+ 141us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:57:18.976415 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:57:18.978625 (+ 2210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:57:18.979749 (+ 1124us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:57:18.982418 (+ 2669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:57:18.983087 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:57:18.983228 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:57:18.988761 (+ 5533us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:57:18.988770 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:57:18.988777 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:57:18.988825 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:57:18.992382 (+ 3557us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:57:18.993606 (+ 1224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:57:18.993614 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:57:18.993621 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:57:18.993812 (+ 191us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:57:18.994400 (+ 588us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:57:18.994407 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:57:18.994410 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:57:18.994691 (+ 281us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:57:18.994868 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0806 04:57:18.995613 (+ 745us) server_negotiation.cc:299] Negotiation successful
0806 04:57:18.995836 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260806 04:57:18.997524 11620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:57:18.972966 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:34519 (local address 127.9.96.66:46585)
0806 04:57:18.973898 (+ 932us) negotiation.cc:107] Waiting for socket to connect
0806 04:57:18.973936 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0806 04:57:18.975852 (+ 1916us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:57:18.976871 (+ 1019us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:57:18.976882 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:57:18.977623 (+ 741us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:57:18.978446 (+ 823us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:57:18.978463 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:57:18.979925 (+ 1462us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:57:18.979931 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:57:18.982232 (+ 2301us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:57:18.982253 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:57:18.983341 (+ 1088us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:57:18.983350 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:57:18.983655 (+ 305us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:57:18.984419 (+ 764us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:57:18.984459 (+ 40us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:57:18.988459 (+ 4000us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 04:57:18.992581 (+ 4122us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:57:18.992594 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:57:18.992615 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:57:18.993427 (+ 812us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:57:18.993971 (+ 544us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:57:18.993982 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:57:18.993986 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:57:18.994259 (+ 273us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:57:18.994838 (+ 579us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 04:57:18.994845 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 04:57:18.995200 (+ 355us) client_negotiation.cc:769] Sending connection context
0806 04:57:18.995521 (+ 321us) client_negotiation.cc:240] Negotiation successful
0806 04:57:18.996110 (+ 589us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":643,"thread_start_us":409,"threads_started":1}
I20260806 04:57:19.019086 11412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:57:19.003457 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.96.66:34313 (local address 127.9.96.126:34519)
0806 04:57:19.003674 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0806 04:57:19.003679 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 04:57:19.004582 (+ 903us) server_negotiation.cc:407] Connection header received
0806 04:57:19.004868 (+ 286us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:57:19.004876 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:57:19.004965 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:57:19.005081 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:57:19.006572 (+ 1491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:57:19.007572 (+ 1000us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:57:19.009590 (+ 2018us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:57:19.010189 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:57:19.010311 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:57:19.012915 (+ 2604us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:57:19.012922 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:57:19.012926 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:57:19.012966 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:57:19.016080 (+ 3114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:57:19.017110 (+ 1030us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:57:19.017115 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:57:19.017118 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:57:19.017248 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:57:19.017627 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:57:19.017632 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:57:19.017635 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:57:19.017950 (+ 315us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:57:19.018109 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0806 04:57:19.018512 (+ 403us) server_negotiation.cc:299] Negotiation successful
0806 04:57:19.018733 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260806 04:57:19.019230 11709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:57:19.003464 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:34519 (local address 127.9.96.66:34313)
0806 04:57:19.004446 (+ 982us) negotiation.cc:107] Waiting for socket to connect
0806 04:57:19.004474 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0806 04:57:19.004635 (+ 161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:57:19.005127 (+ 492us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:57:19.005132 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:57:19.005870 (+ 738us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:57:19.006361 (+ 491us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:57:19.006381 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:57:19.007764 (+ 1383us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:57:19.007767 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:57:19.009373 (+ 1606us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:57:19.009394 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:57:19.010575 (+ 1181us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:57:19.010583 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:57:19.010797 (+ 214us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:57:19.011527 (+ 730us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:57:19.011557 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:57:19.012724 (+ 1167us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 04:57:19.016279 (+ 3555us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:57:19.016289 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:57:19.016293 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:57:19.016969 (+ 676us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:57:19.017367 (+ 398us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:57:19.017374 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:57:19.017377 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:57:19.017514 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:57:19.018075 (+ 561us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 04:57:19.018080 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 04:57:19.018268 (+ 188us) client_negotiation.cc:769] Sending connection context
0806 04:57:19.018421 (+ 153us) client_negotiation.cc:240] Negotiation successful
0806 04:57:19.018744 (+ 323us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":744,"thread_start_us":559,"threads_started":1}
I20260806 04:57:19.024998 11786 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.96.66:42211 every 8 connection(s)
I20260806 04:57:19.024994 11582 rpc_server.cc:307] RPC server started. Bound to: 127.9.96.66:42211
I20260806 04:57:19.026218 11582 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785992227050689-9601-0/minicluster-data/ts-1/data/info.pb
I20260806 04:57:19.027060 9601 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu as pid 11582
I20260806 04:57:19.027298 9601 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785992227050689-9601-0/minicluster-data/ts-1/wal/instance
I20260806 04:57:19.027916 9601 external_mini_cluster.cc:1428] Setting key 6376bb6701aacfbd1467808d7723798e
I20260806 04:57:19.029258 11788 heartbeater.cc:347] Connected to a master server at 127.9.96.126:34519
I20260806 04:57:19.029492 11788 heartbeater.cc:464] Registering TS with master...
I20260806 04:57:19.030319 11788 heartbeater.cc:511] Master 127.9.96.126:34519 requested a full tablet report, sending...
I20260806 04:57:19.031693 11263 ts_manager.cc:194] Registered new tserver with Master: 3c92b56d4ab3443b9c85bdabf4a72977 (127.9.96.66:42211)
I20260806 04:57:19.032542 11263 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.96.66@KRBTEST.COM'} at 127.9.96.66:46585
WARNING: no policy specified for kudu/127.9.96.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.96.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.96.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.67.keytab.
Entry for principal kudu/127.9.96.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.67.keytab.
I20260806 04:57:19.070242 9601 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
/tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785992227050689-9601-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785992227050689-9601-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785992227050689-9601-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785992227050689-9601-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.96.67:0
--local_ip_for_outbound_sockets=127.9.96.67
--webserver_interface=127.9.96.67
--webserver_port=0
--tserver_master_addrs=127.9.96.126:34519
--keytab_file=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.67.keytab
--principal=kudu/127.9.96.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.96.84:41399
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785992227050689-9601-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785992227050689-9601-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785992227050689-9601-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 04:57:19.393123 11792 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 04:57:19.393424 11792 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 04:57:19.393596 11792 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 04:57:19.400743 11792 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 04:57:19.400879 11792 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 04:57:19.401034 11792 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 04:57:19.401173 11792 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.96.67
I20260806 04:57:19.410497 11792 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.96.84:41399
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785992227050689-9601-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785992227050689-9601-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.67.keytab
--principal=kudu/127.9.96.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.96.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785992227050689-9601-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.96.67
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.9.96.126:34519
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.96.67
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785992227050689-9601-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 03:56:12 UTC on ccycloud.quasar-guijbm.root.comops.site
build id 4569618
ASAN enabled
I20260806 04:57:19.411686 11792 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 04:57:19.412920 11792 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 04:57:19.426242 11801 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:57:19.426203 11806 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:57:19.426798 11803 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:57:19.427222 11792 server_base.cc:1034] running on GCE node
I20260806 04:57:19.427832 11792 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 04:57:19.428634 11792 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 04:57:19.429877 11792 hybrid_clock.cc:648] HybridClock initialized: now 1785992239429810 us; error 42 us; skew 500 ppm
I20260806 04:57:19.430159 11792 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 04:57:19 dist-test-slave-x4qh krb5kdc[11195](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992239, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.96.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 04:57:19.433243 11792 init.cc:377] Logged in from keytab as kudu/127.9.96.67@KRBTEST.COM (short username kudu)
I20260806 04:57:19.434662 11792 webserver.cc:460] Webserver started at http://127.9.96.67:44445/ using document root <none> and password file <none>
I20260806 04:57:19.435453 11792 fs_manager.cc:359] Metadata directory not provided
I20260806 04:57:19.435585 11792 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 04:57:19.435917 11792 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 04:57:19.438757 11792 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785992227050689-9601-0/minicluster-data/ts-2/data/instance:
uuid: "578ae1f534534b3aa0585dcbd0781ede"
format_stamp: "Formatted at 2026-08-06 04:57:19 on dist-test-slave-x4qh"
server_key: "cbb8468eecfe557d9e0827aaf6a78edb"
server_key_iv: "91a127e63575cc917f6b5f04bef6df5a"
server_key_version: "encryptionkey@0"
I20260806 04:57:19.439652 11792 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785992227050689-9601-0/minicluster-data/ts-2/wal/instance:
uuid: "578ae1f534534b3aa0585dcbd0781ede"
format_stamp: "Formatted at 2026-08-06 04:57:19 on dist-test-slave-x4qh"
server_key: "cbb8468eecfe557d9e0827aaf6a78edb"
server_key_iv: "91a127e63575cc917f6b5f04bef6df5a"
server_key_version: "encryptionkey@0"
I20260806 04:57:19.444731 11792 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260806 04:57:19.448128 11815 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 04:57:19.449415 11792 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 04:57:19.449600 11792 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785992227050689-9601-0/minicluster-data/ts-2/data,/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785992227050689-9601-0/minicluster-data/ts-2/wal
uuid: "578ae1f534534b3aa0585dcbd0781ede"
format_stamp: "Formatted at 2026-08-06 04:57:19 on dist-test-slave-x4qh"
server_key: "cbb8468eecfe557d9e0827aaf6a78edb"
server_key_iv: "91a127e63575cc917f6b5f04bef6df5a"
server_key_version: "encryptionkey@0"
I20260806 04:57:19.449810 11792 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785992227050689-9601-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785992227050689-9601-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785992227050689-9601-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 04:57:19.482002 11792 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 04:57:19.485894 11792 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 04:57:19.486214 11792 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 04:57:19.489137 11792 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 04:57:19.489262 11792 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 04:57:19.489408 11792 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 04:57:19.489524 11792 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.001s
Aug 06 04:57:19 dist-test-slave-x4qh krb5kdc[11195](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992239, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.96.67@KRBTEST.COM for kudu/127.9.96.126@KRBTEST.COM
I20260806 04:57:19.515282 11412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:57:19.489555 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.96.67:46605 (local address 127.9.96.126:34519)
0806 04:57:19.489723 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0806 04:57:19.489727 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 04:57:19.491151 (+ 1424us) server_negotiation.cc:407] Connection header received
0806 04:57:19.493294 (+ 2143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:57:19.493301 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:57:19.493410 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:57:19.493536 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:57:19.495898 (+ 2362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:57:19.497236 (+ 1338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:57:19.500044 (+ 2808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:57:19.500796 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:57:19.500966 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:57:19.506538 (+ 5572us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:57:19.506560 (+ 22us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:57:19.506567 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:57:19.506614 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:57:19.510640 (+ 4026us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:57:19.511600 (+ 960us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:57:19.511605 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:57:19.511608 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:57:19.511828 (+ 220us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:57:19.512469 (+ 641us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:57:19.512475 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:57:19.512479 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:57:19.512865 (+ 386us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:57:19.513043 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0806 04:57:19.514144 (+ 1101us) server_negotiation.cc:299] Negotiation successful
0806 04:57:19.514456 (+ 312us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260806 04:57:19.516119 11824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:57:19.490160 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:34519 (local address 127.9.96.67:46605)
0806 04:57:19.490977 (+ 817us) negotiation.cc:107] Waiting for socket to connect
0806 04:57:19.491016 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0806 04:57:19.492909 (+ 1893us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:57:19.493766 (+ 857us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:57:19.493782 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:57:19.494614 (+ 832us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:57:19.495631 (+ 1017us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:57:19.495654 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:57:19.497445 (+ 1791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:57:19.497454 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:57:19.499871 (+ 2417us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:57:19.499894 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:57:19.501083 (+ 1189us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:57:19.501093 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:57:19.501424 (+ 331us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:57:19.502251 (+ 827us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:57:19.502288 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:57:19.506262 (+ 3974us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 04:57:19.510785 (+ 4523us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:57:19.510793 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:57:19.510810 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:57:19.511438 (+ 628us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:57:19.511969 (+ 531us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:57:19.511977 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:57:19.511984 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:57:19.512285 (+ 301us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:57:19.513123 (+ 838us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 04:57:19.513131 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 04:57:19.513548 (+ 417us) client_negotiation.cc:769] Sending connection context
0806 04:57:19.513875 (+ 327us) client_negotiation.cc:240] Negotiation successful
0806 04:57:19.514403 (+ 528us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":574,"thread_start_us":382,"threads_started":1}
I20260806 04:57:19.534135 11575 heartbeater.cc:503] Master 127.9.96.126:34519 was elected leader, sending a full tablet report...
I20260806 04:57:19.541863 11909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:57:19.528177 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:34519 (local address 127.9.96.67:52843)
0806 04:57:19.528838 (+ 661us) negotiation.cc:107] Waiting for socket to connect
0806 04:57:19.528853 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 04:57:19.529014 (+ 161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:57:19.529352 (+ 338us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:57:19.529356 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:57:19.529781 (+ 425us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:57:19.530047 (+ 266us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:57:19.530055 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:57:19.530976 (+ 921us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:57:19.530981 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:57:19.532522 (+ 1541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:57:19.532537 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:57:19.533762 (+ 1225us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:57:19.533769 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:57:19.533979 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:57:19.534767 (+ 788us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:57:19.534789 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:57:19.535942 (+ 1153us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 04:57:19.538973 (+ 3031us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:57:19.538978 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:57:19.538982 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:57:19.539353 (+ 371us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:57:19.539765 (+ 412us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:57:19.539769 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:57:19.539773 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:57:19.539901 (+ 128us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:57:19.540523 (+ 622us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 04:57:19.540528 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 04:57:19.540766 (+ 238us) client_negotiation.cc:769] Sending connection context
0806 04:57:19.540937 (+ 171us) client_negotiation.cc:240] Negotiation successful
0806 04:57:19.541310 (+ 373us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":519,"thread_start_us":366,"threads_started":1}
I20260806 04:57:19.542778 11412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:57:19.528115 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.96.67:52843 (local address 127.9.96.126:34519)
0806 04:57:19.528308 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0806 04:57:19.528314 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 04:57:19.528940 (+ 626us) server_negotiation.cc:407] Connection header received
0806 04:57:19.529147 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:57:19.529151 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:57:19.529227 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:57:19.529329 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:57:19.530189 (+ 860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:57:19.530846 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:57:19.532782 (+ 1936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:57:19.533522 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:57:19.533687 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:57:19.536133 (+ 2446us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:57:19.536143 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:57:19.536146 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:57:19.536174 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:57:19.538831 (+ 2657us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:57:19.539515 (+ 684us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:57:19.539521 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:57:19.539524 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:57:19.539646 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:57:19.540022 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:57:19.540030 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:57:19.540034 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:57:19.540402 (+ 368us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:57:19.540553 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0806 04:57:19.542220 (+ 1667us) server_negotiation.cc:299] Negotiation successful
0806 04:57:19.542494 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260806 04:57:19.554461 11792 rpc_server.cc:307] RPC server started. Bound to: 127.9.96.67:32807
I20260806 04:57:19.554553 11982 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.96.67:32807 every 8 connection(s)
I20260806 04:57:19.555759 11792 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785992227050689-9601-0/minicluster-data/ts-2/data/info.pb
I20260806 04:57:19.557713 11983 heartbeater.cc:347] Connected to a master server at 127.9.96.126:34519
I20260806 04:57:19.557950 11983 heartbeater.cc:464] Registering TS with master...
I20260806 04:57:19.558643 9601 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu as pid 11792
I20260806 04:57:19.558694 11983 heartbeater.cc:511] Master 127.9.96.126:34519 requested a full tablet report, sending...
I20260806 04:57:19.558846 9601 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785992227050689-9601-0/minicluster-data/ts-2/wal/instance
I20260806 04:57:19.559430 9601 external_mini_cluster.cc:1428] Setting key e1926ca4c6d47f57b4220d80dc8da4f1
I20260806 04:57:19.560153 11263 ts_manager.cc:194] Registered new tserver with Master: 578ae1f534534b3aa0585dcbd0781ede (127.9.96.67:32807)
I20260806 04:57:19.561318 11263 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.96.67@KRBTEST.COM'} at 127.9.96.67:46605
I20260806 04:57:19.562467 9601 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 06 04:57:19 dist-test-slave-x4qh krb5kdc[11195](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992239, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 06 04:57:19 dist-test-slave-x4qh krb5kdc[11195](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992239, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.9.96.126@KRBTEST.COM
I20260806 04:57:19.610664 11412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:57:19.596859 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51904 (local address 127.9.96.126:34519)
0806 04:57:19.597087 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0806 04:57:19.597095 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 04:57:19.597286 (+ 191us) server_negotiation.cc:407] Connection header received
0806 04:57:19.597505 (+ 219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:57:19.597509 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:57:19.597573 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:57:19.597679 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:57:19.599151 (+ 1472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:57:19.600041 (+ 890us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:57:19.601326 (+ 1285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:57:19.601750 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:57:19.601886 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:57:19.605648 (+ 3762us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:57:19.605654 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:57:19.605657 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:57:19.605690 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:57:19.608161 (+ 2471us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:57:19.608774 (+ 613us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:57:19.608780 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:57:19.608783 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:57:19.608916 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:57:19.609280 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:57:19.609287 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:57:19.609290 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:57:19.609551 (+ 261us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:57:19.609805 (+ 254us) server_negotiation.cc:1035] Waiting for connection context
0806 04:57:19.610082 (+ 277us) server_negotiation.cc:299] Negotiation successful
0806 04:57:19.610310 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260806 04:57:19.614627 11264 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:51904:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 04:57:19.622009 11264 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 04:57:19.654286 12006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:57:19.634230 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33626 (local address 127.9.96.66:42211)
0806 04:57:19.644368 (+ 10138us) server_negotiation.cc:206] Beginning negotiation
0806 04:57:19.644395 (+ 27us) server_negotiation.cc:399] Waiting for connection header
0806 04:57:19.644756 (+ 361us) server_negotiation.cc:407] Connection header received
0806 04:57:19.645594 (+ 838us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:57:19.645602 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:57:19.645833 (+ 231us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:57:19.646007 (+ 174us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 04:57:19.647070 (+ 1063us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:57:19.648638 (+ 1568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:57:19.651364 (+ 2726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:57:19.652822 (+ 1458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:57:19.652962 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:57:19.653158 (+ 196us) server_negotiation.cc:1035] Waiting for connection context
0806 04:57:19.653795 (+ 637us) server_negotiation.cc:299] Negotiation successful
0806 04:57:19.653990 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9953,"thread_start_us":10167,"threads_started":1}
I20260806 04:57:19.654335 12004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:57:19.633839 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33250 (local address 127.9.96.67:32807)
0806 04:57:19.645657 (+ 11818us) server_negotiation.cc:206] Beginning negotiation
0806 04:57:19.645662 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 04:57:19.645686 (+ 24us) server_negotiation.cc:407] Connection header received
0806 04:57:19.645761 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:57:19.645766 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:57:19.646025 (+ 259us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:57:19.646235 (+ 210us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 04:57:19.647084 (+ 849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:57:19.648756 (+ 1672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:57:19.650684 (+ 1928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:57:19.652313 (+ 1629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:57:19.653525 (+ 1212us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:57:19.653755 (+ 230us) server_negotiation.cc:1035] Waiting for connection context
0806 04:57:19.653827 (+ 72us) server_negotiation.cc:299] Negotiation successful
0806 04:57:19.653987 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11662,"thread_start_us":11444,"threads_started":1}
I20260806 04:57:19.654546 12011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:57:19.636033 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40950 (local address 127.9.96.65:39511)
0806 04:57:19.645181 (+ 9148us) server_negotiation.cc:206] Beginning negotiation
0806 04:57:19.645187 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 04:57:19.645210 (+ 23us) server_negotiation.cc:407] Connection header received
0806 04:57:19.645403 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:57:19.645411 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:57:19.645664 (+ 253us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:57:19.645883 (+ 219us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 04:57:19.646868 (+ 985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:57:19.648204 (+ 1336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:57:19.651031 (+ 2827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:57:19.652569 (+ 1538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:57:19.652698 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:57:19.652933 (+ 235us) server_negotiation.cc:1035] Waiting for connection context
0806 04:57:19.653936 (+ 1003us) server_negotiation.cc:299] Negotiation successful
0806 04:57:19.654101 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9012,"thread_start_us":8876,"threads_started":1}
I20260806 04:57:19.654807 12005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:57:19.635469 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.65:39511 (local address 127.0.0.1:40950)
0806 04:57:19.645067 (+ 9598us) negotiation.cc:107] Waiting for socket to connect
0806 04:57:19.645085 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 04:57:19.645216 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:57:19.645911 (+ 695us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:57:19.645918 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:57:19.645946 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 04:57:19.646464 (+ 518us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:57:19.646481 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:57:19.648343 (+ 1862us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:57:19.648350 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:57:19.650858 (+ 2508us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:57:19.650903 (+ 45us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:57:19.652871 (+ 1968us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:57:19.652882 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:57:19.653083 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:57:19.653116 (+ 33us) client_negotiation.cc:769] Sending connection context
0806 04:57:19.653239 (+ 123us) client_negotiation.cc:240] Negotiation successful
0806 04:57:19.653397 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9486,"mutex_wait_us":45,"thread_start_us":8743,"threads_started":1}
I20260806 04:57:19.654852 12002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:57:19.633554 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.67:32807 (local address 127.0.0.1:33250)
0806 04:57:19.634589 (+ 1035us) negotiation.cc:107] Waiting for socket to connect
0806 04:57:19.634625 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0806 04:57:19.634812 (+ 187us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:57:19.646240 (+ 11428us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:57:19.646245 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:57:19.646299 (+ 54us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 04:57:19.646827 (+ 528us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:57:19.646847 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:57:19.648878 (+ 2031us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:57:19.648883 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:57:19.650512 (+ 1629us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:57:19.650535 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:57:19.652530 (+ 1995us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:57:19.652537 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:57:19.652731 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:57:19.652788 (+ 57us) client_negotiation.cc:769] Sending connection context
0806 04:57:19.652903 (+ 115us) client_negotiation.cc:240] Negotiation successful
0806 04:57:19.653116 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":722,"spinlock_wait_cycles":2816,"thread_start_us":1091,"threads_started":1}
I20260806 04:57:19.655212 12008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:57:19.633959 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.66:42211 (local address 127.0.0.1:33626)
0806 04:57:19.644583 (+ 10624us) negotiation.cc:107] Waiting for socket to connect
0806 04:57:19.644607 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0806 04:57:19.645403 (+ 796us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:57:19.646240 (+ 837us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:57:19.646245 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:57:19.646281 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 04:57:19.646862 (+ 581us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:57:19.646878 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:57:19.648784 (+ 1906us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:57:19.648789 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:57:19.651213 (+ 2424us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:57:19.651227 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:57:19.653334 (+ 2107us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:57:19.653343 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:57:19.653523 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:57:19.653560 (+ 37us) client_negotiation.cc:769] Sending connection context
0806 04:57:19.653651 (+ 91us) client_negotiation.cc:240] Negotiation successful
0806 04:57:19.654636 (+ 985us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10456,"mutex_wait_us":134,"spinlock_wait_cycles":896,"thread_start_us":10833,"threads_started":1}
I20260806 04:57:19.658666 11881 tablet_service.cc:1467] Processing CreateTablet for tablet 364b1d62b33d45379c85dc9c95b68435 (DEFAULT_TABLE table=test-table [id=48073d66ee7e45f099e15cf0bdb229a7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 04:57:19.658783 11677 tablet_service.cc:1467] Processing CreateTablet for tablet 364b1d62b33d45379c85dc9c95b68435 (DEFAULT_TABLE table=test-table [id=48073d66ee7e45f099e15cf0bdb229a7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 04:57:19.658782 11475 tablet_service.cc:1467] Processing CreateTablet for tablet 364b1d62b33d45379c85dc9c95b68435 (DEFAULT_TABLE table=test-table [id=48073d66ee7e45f099e15cf0bdb229a7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 04:57:19.660164 11475 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 364b1d62b33d45379c85dc9c95b68435. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 04:57:19.660180 11677 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 364b1d62b33d45379c85dc9c95b68435. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 04:57:19.660542 11881 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 364b1d62b33d45379c85dc9c95b68435. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 04:57:19.671941 12012 tablet_bootstrap.cc:492] T 364b1d62b33d45379c85dc9c95b68435 P 578ae1f534534b3aa0585dcbd0781ede: Bootstrap starting.
I20260806 04:57:19.674675 12014 tablet_bootstrap.cc:492] T 364b1d62b33d45379c85dc9c95b68435 P bb34b1fe2b6a439fb22a40a0e22d9607: Bootstrap starting.
I20260806 04:57:19.676227 12015 tablet_bootstrap.cc:492] T 364b1d62b33d45379c85dc9c95b68435 P 3c92b56d4ab3443b9c85bdabf4a72977: Bootstrap starting.
I20260806 04:57:19.677402 12012 tablet_bootstrap.cc:654] T 364b1d62b33d45379c85dc9c95b68435 P 578ae1f534534b3aa0585dcbd0781ede: Neither blocks nor log segments found. Creating new log.
I20260806 04:57:19.678114 12014 tablet_bootstrap.cc:654] T 364b1d62b33d45379c85dc9c95b68435 P bb34b1fe2b6a439fb22a40a0e22d9607: Neither blocks nor log segments found. Creating new log.
I20260806 04:57:19.678725 12012 log.cc:824] T 364b1d62b33d45379c85dc9c95b68435 P 578ae1f534534b3aa0585dcbd0781ede: Log is configured to *not* fsync() on all Append() calls
I20260806 04:57:19.678915 12015 tablet_bootstrap.cc:654] T 364b1d62b33d45379c85dc9c95b68435 P 3c92b56d4ab3443b9c85bdabf4a72977: Neither blocks nor log segments found. Creating new log.
I20260806 04:57:19.679342 12014 log.cc:824] T 364b1d62b33d45379c85dc9c95b68435 P bb34b1fe2b6a439fb22a40a0e22d9607: Log is configured to *not* fsync() on all Append() calls
I20260806 04:57:19.680168 12015 log.cc:824] T 364b1d62b33d45379c85dc9c95b68435 P 3c92b56d4ab3443b9c85bdabf4a72977: Log is configured to *not* fsync() on all Append() calls
I20260806 04:57:19.682013 12012 tablet_bootstrap.cc:492] T 364b1d62b33d45379c85dc9c95b68435 P 578ae1f534534b3aa0585dcbd0781ede: No bootstrap required, opened a new log
I20260806 04:57:19.682277 12014 tablet_bootstrap.cc:492] T 364b1d62b33d45379c85dc9c95b68435 P bb34b1fe2b6a439fb22a40a0e22d9607: No bootstrap required, opened a new log
I20260806 04:57:19.682300 12012 ts_tablet_manager.cc:1397] T 364b1d62b33d45379c85dc9c95b68435 P 578ae1f534534b3aa0585dcbd0781ede: Time spent bootstrapping tablet: real 0.011s user 0.003s sys 0.004s
I20260806 04:57:19.682541 12014 ts_tablet_manager.cc:1397] T 364b1d62b33d45379c85dc9c95b68435 P bb34b1fe2b6a439fb22a40a0e22d9607: Time spent bootstrapping tablet: real 0.008s user 0.003s sys 0.003s
I20260806 04:57:19.682689 12015 tablet_bootstrap.cc:492] T 364b1d62b33d45379c85dc9c95b68435 P 3c92b56d4ab3443b9c85bdabf4a72977: No bootstrap required, opened a new log
I20260806 04:57:19.682945 12015 ts_tablet_manager.cc:1397] T 364b1d62b33d45379c85dc9c95b68435 P 3c92b56d4ab3443b9c85bdabf4a72977: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.005s
I20260806 04:57:19.690038 12015 raft_consensus.cc:348] T 364b1d62b33d45379c85dc9c95b68435 P 3c92b56d4ab3443b9c85bdabf4a72977 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "578ae1f534534b3aa0585dcbd0781ede" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 32807 } } peers { permanent_uuid: "bb34b1fe2b6a439fb22a40a0e22d9607" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 39511 } } peers { permanent_uuid: "3c92b56d4ab3443b9c85bdabf4a72977" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 42211 } }
I20260806 04:57:19.690387 12015 raft_consensus.cc:374] T 364b1d62b33d45379c85dc9c95b68435 P 3c92b56d4ab3443b9c85bdabf4a72977 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 04:57:19.690519 12015 raft_consensus.cc:729] T 364b1d62b33d45379c85dc9c95b68435 P 3c92b56d4ab3443b9c85bdabf4a72977 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3c92b56d4ab3443b9c85bdabf4a72977, State: Initialized, Role: FOLLOWER
I20260806 04:57:19.691116 12015 consensus_queue.cc:260] T 364b1d62b33d45379c85dc9c95b68435 P 3c92b56d4ab3443b9c85bdabf4a72977 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "578ae1f534534b3aa0585dcbd0781ede" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 32807 } } peers { permanent_uuid: "bb34b1fe2b6a439fb22a40a0e22d9607" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 39511 } } peers { permanent_uuid: "3c92b56d4ab3443b9c85bdabf4a72977" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 42211 } }
I20260806 04:57:19.692198 11788 heartbeater.cc:503] Master 127.9.96.126:34519 was elected leader, sending a full tablet report...
I20260806 04:57:19.692687 12015 ts_tablet_manager.cc:1428] T 364b1d62b33d45379c85dc9c95b68435 P 3c92b56d4ab3443b9c85bdabf4a72977: Time spent starting tablet: real 0.010s user 0.010s sys 0.000s
I20260806 04:57:19.693576 12014 raft_consensus.cc:348] T 364b1d62b33d45379c85dc9c95b68435 P bb34b1fe2b6a439fb22a40a0e22d9607 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "578ae1f534534b3aa0585dcbd0781ede" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 32807 } } peers { permanent_uuid: "bb34b1fe2b6a439fb22a40a0e22d9607" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 39511 } } peers { permanent_uuid: "3c92b56d4ab3443b9c85bdabf4a72977" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 42211 } }
I20260806 04:57:19.694069 12014 raft_consensus.cc:374] T 364b1d62b33d45379c85dc9c95b68435 P bb34b1fe2b6a439fb22a40a0e22d9607 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 04:57:19.694177 12014 raft_consensus.cc:729] T 364b1d62b33d45379c85dc9c95b68435 P bb34b1fe2b6a439fb22a40a0e22d9607 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bb34b1fe2b6a439fb22a40a0e22d9607, State: Initialized, Role: FOLLOWER
I20260806 04:57:19.693987 12012 raft_consensus.cc:348] T 364b1d62b33d45379c85dc9c95b68435 P 578ae1f534534b3aa0585dcbd0781ede [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "578ae1f534534b3aa0585dcbd0781ede" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 32807 } } peers { permanent_uuid: "bb34b1fe2b6a439fb22a40a0e22d9607" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 39511 } } peers { permanent_uuid: "3c92b56d4ab3443b9c85bdabf4a72977" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 42211 } }
I20260806 04:57:19.694422 12012 raft_consensus.cc:374] T 364b1d62b33d45379c85dc9c95b68435 P 578ae1f534534b3aa0585dcbd0781ede [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 04:57:19.694543 12012 raft_consensus.cc:729] T 364b1d62b33d45379c85dc9c95b68435 P 578ae1f534534b3aa0585dcbd0781ede [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 578ae1f534534b3aa0585dcbd0781ede, State: Initialized, Role: FOLLOWER
I20260806 04:57:19.694887 12014 consensus_queue.cc:260] T 364b1d62b33d45379c85dc9c95b68435 P bb34b1fe2b6a439fb22a40a0e22d9607 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "578ae1f534534b3aa0585dcbd0781ede" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 32807 } } peers { permanent_uuid: "bb34b1fe2b6a439fb22a40a0e22d9607" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 39511 } } peers { permanent_uuid: "3c92b56d4ab3443b9c85bdabf4a72977" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 42211 } }
I20260806 04:57:19.695125 12012 consensus_queue.cc:260] T 364b1d62b33d45379c85dc9c95b68435 P 578ae1f534534b3aa0585dcbd0781ede [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "578ae1f534534b3aa0585dcbd0781ede" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 32807 } } peers { permanent_uuid: "bb34b1fe2b6a439fb22a40a0e22d9607" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 39511 } } peers { permanent_uuid: "3c92b56d4ab3443b9c85bdabf4a72977" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 42211 } }
I20260806 04:57:19.696730 12014 ts_tablet_manager.cc:1428] T 364b1d62b33d45379c85dc9c95b68435 P bb34b1fe2b6a439fb22a40a0e22d9607: Time spent starting tablet: real 0.014s user 0.009s sys 0.006s
I20260806 04:57:19.697636 11983 heartbeater.cc:503] Master 127.9.96.126:34519 was elected leader, sending a full tablet report...
I20260806 04:57:19.698787 12012 ts_tablet_manager.cc:1428] T 364b1d62b33d45379c85dc9c95b68435 P 578ae1f534534b3aa0585dcbd0781ede: Time spent starting tablet: real 0.016s user 0.016s sys 0.000s
W20260806 04:57:19.776351 11576 tablet.cc:2369] T 364b1d62b33d45379c85dc9c95b68435 P bb34b1fe2b6a439fb22a40a0e22d9607: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 04:57:19.778127 11789 tablet.cc:2369] T 364b1d62b33d45379c85dc9c95b68435 P 3c92b56d4ab3443b9c85bdabf4a72977: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 04:57:19.807454 11984 tablet.cc:2369] T 364b1d62b33d45379c85dc9c95b68435 P 578ae1f534534b3aa0585dcbd0781ede: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 04:57:19.978237 12025 raft_consensus.cc:482] T 364b1d62b33d45379c85dc9c95b68435 P 578ae1f534534b3aa0585dcbd0781ede [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 04:57:19.978603 12025 raft_consensus.cc:504] T 364b1d62b33d45379c85dc9c95b68435 P 578ae1f534534b3aa0585dcbd0781ede [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "578ae1f534534b3aa0585dcbd0781ede" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 32807 } } peers { permanent_uuid: "bb34b1fe2b6a439fb22a40a0e22d9607" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 39511 } } peers { permanent_uuid: "3c92b56d4ab3443b9c85bdabf4a72977" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 42211 } }
I20260806 04:57:19.980401 12025 leader_election.cc:288] T 364b1d62b33d45379c85dc9c95b68435 P 578ae1f534534b3aa0585dcbd0781ede [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bb34b1fe2b6a439fb22a40a0e22d9607 (127.9.96.65:39511), 3c92b56d4ab3443b9c85bdabf4a72977 (127.9.96.66:42211)
I20260806 04:57:19.987524 12011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:57:19.980856 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.96.67:44725 (local address 127.9.96.65:39511)
0806 04:57:19.981070 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0806 04:57:19.981075 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 04:57:19.981096 (+ 21us) server_negotiation.cc:407] Connection header received
0806 04:57:19.981333 (+ 237us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:57:19.981341 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:57:19.981443 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:57:19.981543 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 04:57:19.982041 (+ 498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:57:19.983007 (+ 966us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:57:19.985478 (+ 2471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:57:19.986664 (+ 1186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:57:19.986852 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:57:19.986962 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0806 04:57:19.987148 (+ 186us) server_negotiation.cc:299] Negotiation successful
0806 04:57:19.987283 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260806 04:57:19.987566 11824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:57:19.980788 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.65:39511 (local address 127.9.96.67:44725)
0806 04:57:19.980973 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0806 04:57:19.980993 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0806 04:57:19.981120 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:57:19.981610 (+ 490us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:57:19.981614 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:57:19.981643 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 04:57:19.981897 (+ 254us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:57:19.981906 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:57:19.983134 (+ 1228us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:57:19.983138 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:57:19.985287 (+ 2149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:57:19.985303 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:57:19.986856 (+ 1553us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:57:19.986860 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:57:19.986989 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:57:19.987012 (+ 23us) client_negotiation.cc:769] Sending connection context
0806 04:57:19.987086 (+ 74us) client_negotiation.cc:240] Negotiation successful
0806 04:57:19.987215 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":54,"spinlock_wait_cycles":19712}
I20260806 04:57:19.988165 12006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:57:19.980897 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.96.67:54879 (local address 127.9.96.66:42211)
0806 04:57:19.981180 (+ 283us) server_negotiation.cc:206] Beginning negotiation
0806 04:57:19.981187 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 04:57:19.981645 (+ 458us) server_negotiation.cc:407] Connection header received
0806 04:57:19.981862 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:57:19.981865 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:57:19.981925 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:57:19.982040 (+ 115us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 04:57:19.982600 (+ 560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:57:19.983579 (+ 979us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:57:19.985838 (+ 2259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:57:19.987089 (+ 1251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:57:19.987245 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:57:19.987345 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0806 04:57:19.987754 (+ 409us) server_negotiation.cc:299] Negotiation successful
0806 04:57:19.987897 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":113}
I20260806 04:57:19.988669 12027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:57:19.980802 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.66:42211 (local address 127.9.96.67:54879)
0806 04:57:19.981497 (+ 695us) negotiation.cc:107] Waiting for socket to connect
0806 04:57:19.981540 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0806 04:57:19.981728 (+ 188us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:57:19.982094 (+ 366us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:57:19.982098 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:57:19.982120 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 04:57:19.982435 (+ 315us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:57:19.982442 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:57:19.983752 (+ 1310us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:57:19.983760 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:57:19.985683 (+ 1923us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:57:19.985693 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:57:19.987332 (+ 1639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:57:19.987338 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:57:19.987529 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:57:19.987571 (+ 42us) client_negotiation.cc:769] Sending connection context
0806 04:57:19.988273 (+ 702us) client_negotiation.cc:240] Negotiation successful
0806 04:57:19.988407 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":605,"mutex_wait_us":168,"spinlock_wait_cycles":22784,"thread_start_us":281,"threads_started":1}
I20260806 04:57:19.988847 11501 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "364b1d62b33d45379c85dc9c95b68435" candidate_uuid: "578ae1f534534b3aa0585dcbd0781ede" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb34b1fe2b6a439fb22a40a0e22d9607" is_pre_election: true
I20260806 04:57:19.989372 11501 raft_consensus.cc:2454] T 364b1d62b33d45379c85dc9c95b68435 P bb34b1fe2b6a439fb22a40a0e22d9607 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 578ae1f534534b3aa0585dcbd0781ede in term 0.
I20260806 04:57:19.989581 11718 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "364b1d62b33d45379c85dc9c95b68435" candidate_uuid: "578ae1f534534b3aa0585dcbd0781ede" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c92b56d4ab3443b9c85bdabf4a72977" is_pre_election: true
I20260806 04:57:19.990087 11816 leader_election.cc:302] T 364b1d62b33d45379c85dc9c95b68435 P 578ae1f534534b3aa0585dcbd0781ede [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 578ae1f534534b3aa0585dcbd0781ede, bb34b1fe2b6a439fb22a40a0e22d9607; no voters:
I20260806 04:57:19.990157 11718 raft_consensus.cc:2454] T 364b1d62b33d45379c85dc9c95b68435 P 3c92b56d4ab3443b9c85bdabf4a72977 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 578ae1f534534b3aa0585dcbd0781ede in term 0.
I20260806 04:57:19.990501 12025 raft_consensus.cc:2781] T 364b1d62b33d45379c85dc9c95b68435 P 578ae1f534534b3aa0585dcbd0781ede [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 04:57:19.990657 12025 raft_consensus.cc:482] T 364b1d62b33d45379c85dc9c95b68435 P 578ae1f534534b3aa0585dcbd0781ede [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 04:57:19.990800 12025 raft_consensus.cc:3037] T 364b1d62b33d45379c85dc9c95b68435 P 578ae1f534534b3aa0585dcbd0781ede [term 0 FOLLOWER]: Advancing to term 1
I20260806 04:57:19.993156 12025 raft_consensus.cc:504] T 364b1d62b33d45379c85dc9c95b68435 P 578ae1f534534b3aa0585dcbd0781ede [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "578ae1f534534b3aa0585dcbd0781ede" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 32807 } } peers { permanent_uuid: "bb34b1fe2b6a439fb22a40a0e22d9607" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 39511 } } peers { permanent_uuid: "3c92b56d4ab3443b9c85bdabf4a72977" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 42211 } }
I20260806 04:57:19.993888 12025 leader_election.cc:288] T 364b1d62b33d45379c85dc9c95b68435 P 578ae1f534534b3aa0585dcbd0781ede [CANDIDATE]: Term 1 election: Requested vote from peers bb34b1fe2b6a439fb22a40a0e22d9607 (127.9.96.65:39511), 3c92b56d4ab3443b9c85bdabf4a72977 (127.9.96.66:42211)
I20260806 04:57:19.994271 11501 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "364b1d62b33d45379c85dc9c95b68435" candidate_uuid: "578ae1f534534b3aa0585dcbd0781ede" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb34b1fe2b6a439fb22a40a0e22d9607"
I20260806 04:57:19.994379 11718 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "364b1d62b33d45379c85dc9c95b68435" candidate_uuid: "578ae1f534534b3aa0585dcbd0781ede" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c92b56d4ab3443b9c85bdabf4a72977"
I20260806 04:57:19.994468 11501 raft_consensus.cc:3037] T 364b1d62b33d45379c85dc9c95b68435 P bb34b1fe2b6a439fb22a40a0e22d9607 [term 0 FOLLOWER]: Advancing to term 1
I20260806 04:57:19.994619 11718 raft_consensus.cc:3037] T 364b1d62b33d45379c85dc9c95b68435 P 3c92b56d4ab3443b9c85bdabf4a72977 [term 0 FOLLOWER]: Advancing to term 1
I20260806 04:57:19.996871 11501 raft_consensus.cc:2454] T 364b1d62b33d45379c85dc9c95b68435 P bb34b1fe2b6a439fb22a40a0e22d9607 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 578ae1f534534b3aa0585dcbd0781ede in term 1.
I20260806 04:57:19.997128 11718 raft_consensus.cc:2454] T 364b1d62b33d45379c85dc9c95b68435 P 3c92b56d4ab3443b9c85bdabf4a72977 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 578ae1f534534b3aa0585dcbd0781ede in term 1.
I20260806 04:57:19.997272 11816 leader_election.cc:302] T 364b1d62b33d45379c85dc9c95b68435 P 578ae1f534534b3aa0585dcbd0781ede [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 578ae1f534534b3aa0585dcbd0781ede, bb34b1fe2b6a439fb22a40a0e22d9607; no voters:
I20260806 04:57:19.997563 12025 raft_consensus.cc:2781] T 364b1d62b33d45379c85dc9c95b68435 P 578ae1f534534b3aa0585dcbd0781ede [term 1 FOLLOWER]: Leader election won for term 1
I20260806 04:57:19.998090 12025 raft_consensus.cc:686] T 364b1d62b33d45379c85dc9c95b68435 P 578ae1f534534b3aa0585dcbd0781ede [term 1 LEADER]: Becoming Leader. State: Replica: 578ae1f534534b3aa0585dcbd0781ede, State: Running, Role: LEADER
I20260806 04:57:19.998665 12025 consensus_queue.cc:237] T 364b1d62b33d45379c85dc9c95b68435 P 578ae1f534534b3aa0585dcbd0781ede [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "578ae1f534534b3aa0585dcbd0781ede" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 32807 } } peers { permanent_uuid: "bb34b1fe2b6a439fb22a40a0e22d9607" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 39511 } } peers { permanent_uuid: "3c92b56d4ab3443b9c85bdabf4a72977" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 42211 } }
I20260806 04:57:20.004029 11264 catalog_manager.cc:5495] T 364b1d62b33d45379c85dc9c95b68435 P 578ae1f534534b3aa0585dcbd0781ede reported cstate change: term changed from 0 to 1, leader changed from <none> to 578ae1f534534b3aa0585dcbd0781ede (127.9.96.67). New cstate: current_term: 1 leader_uuid: "578ae1f534534b3aa0585dcbd0781ede" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "578ae1f534534b3aa0585dcbd0781ede" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 32807 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bb34b1fe2b6a439fb22a40a0e22d9607" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 39511 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3c92b56d4ab3443b9c85bdabf4a72977" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 42211 } health_report { overall_health: UNKNOWN } } }
I20260806 04:57:20.111075 12004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:57:20.105676 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33258 (local address 127.9.96.67:32807)
0806 04:57:20.105869 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0806 04:57:20.105874 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 04:57:20.106032 (+ 158us) server_negotiation.cc:407] Connection header received
0806 04:57:20.106337 (+ 305us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:57:20.106341 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:57:20.106429 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:57:20.106550 (+ 121us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 04:57:20.106919 (+ 369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:57:20.107687 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:57:20.108964 (+ 1277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:57:20.109357 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:57:20.109450 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:57:20.109935 (+ 485us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 04:57:20.110500 (+ 565us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 04:57:20.110616 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0806 04:57:20.110724 (+ 108us) server_negotiation.cc:299] Negotiation successful
0806 04:57:20.110789 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260806 04:57:20.121414 11718 raft_consensus.cc:1261] T 364b1d62b33d45379c85dc9c95b68435 P 3c92b56d4ab3443b9c85bdabf4a72977 [term 1 FOLLOWER]: Refusing update from remote peer 578ae1f534534b3aa0585dcbd0781ede: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 04:57:20.122639 12028 consensus_queue.cc:1035] T 364b1d62b33d45379c85dc9c95b68435 P 578ae1f534534b3aa0585dcbd0781ede [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c92b56d4ab3443b9c85bdabf4a72977" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 42211 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 04:57:20.123226 11501 raft_consensus.cc:1261] T 364b1d62b33d45379c85dc9c95b68435 P bb34b1fe2b6a439fb22a40a0e22d9607 [term 1 FOLLOWER]: Refusing update from remote peer 578ae1f534534b3aa0585dcbd0781ede: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 04:57:20.124115 12028 consensus_queue.cc:1035] T 364b1d62b33d45379c85dc9c95b68435 P 578ae1f534534b3aa0585dcbd0781ede [LEADER]: Connected to new peer: Peer: permanent_uuid: "bb34b1fe2b6a439fb22a40a0e22d9607" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 39511 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 04:57:20.134814 12031 mvcc.cc:205] Tried to move back new op lower bound from 7315424215517097984 to 7315424215039586304. Current Snapshot: MvccSnapshot[applied={T|T < 7315424215517097984}]
I20260806 04:57:20.135725 12034 mvcc.cc:205] Tried to move back new op lower bound from 7315424215517097984 to 7315424215039586304. Current Snapshot: MvccSnapshot[applied={T|T < 7315424215517097984}]
I20260806 04:57:20.156361 11260 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:51904:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 04:57:20.156759 11260 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:51904:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 04:57:20.160697 11260 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d1b98044e21d4333a884f6762082b055: Sending DeleteTablet for 3 replicas of tablet 364b1d62b33d45379c85dc9c95b68435
I20260806 04:57:20.161973 11881 tablet_service.cc:1514] Processing DeleteTablet for tablet 364b1d62b33d45379c85dc9c95b68435 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 04:57:20 UTC) from {username='kudu', principal='kudu/127.9.96.126@KRBTEST.COM'} at 127.0.0.1:33250
I20260806 04:57:20.162038 11475 tablet_service.cc:1514] Processing DeleteTablet for tablet 364b1d62b33d45379c85dc9c95b68435 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 04:57:20 UTC) from {username='kudu', principal='kudu/127.9.96.126@KRBTEST.COM'} at 127.0.0.1:40950
I20260806 04:57:20.163172 11677 tablet_service.cc:1514] Processing DeleteTablet for tablet 364b1d62b33d45379c85dc9c95b68435 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 04:57:20 UTC) from {username='kudu', principal='kudu/127.9.96.126@KRBTEST.COM'} at 127.0.0.1:33626
I20260806 04:57:20.168179 11260 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:51904:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 04:57:20.169337 11260 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 04:57:20.174103 12050 tablet_replica.cc:330] T 364b1d62b33d45379c85dc9c95b68435 P 3c92b56d4ab3443b9c85bdabf4a72977: stopping tablet replica
I20260806 04:57:20.174605 12050 raft_consensus.cc:2229] T 364b1d62b33d45379c85dc9c95b68435 P 3c92b56d4ab3443b9c85bdabf4a72977 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 04:57:20.175083 12050 raft_consensus.cc:2258] T 364b1d62b33d45379c85dc9c95b68435 P 3c92b56d4ab3443b9c85bdabf4a72977 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 04:57:20.176860 12049 tablet_replica.cc:330] T 364b1d62b33d45379c85dc9c95b68435 P 578ae1f534534b3aa0585dcbd0781ede: stopping tablet replica
I20260806 04:57:20.177266 12049 raft_consensus.cc:2229] T 364b1d62b33d45379c85dc9c95b68435 P 578ae1f534534b3aa0585dcbd0781ede [term 1 LEADER]: Raft consensus shutting down.
I20260806 04:57:20.178017 12047 tablet_replica.cc:330] T 364b1d62b33d45379c85dc9c95b68435 P bb34b1fe2b6a439fb22a40a0e22d9607: stopping tablet replica
I20260806 04:57:20.178118 12049 raft_consensus.cc:2258] T 364b1d62b33d45379c85dc9c95b68435 P 578ae1f534534b3aa0585dcbd0781ede [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 04:57:20.178419 12047 raft_consensus.cc:2229] T 364b1d62b33d45379c85dc9c95b68435 P bb34b1fe2b6a439fb22a40a0e22d9607 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 04:57:20.180675 11881 tablet_service.cc:1467] Processing CreateTablet for tablet adfa098aa30244478834002fa731070e (DEFAULT_TABLE table=test-table [id=10a4a48105614cb395d55c24b097deaf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 04:57:20.180545 11677 tablet_service.cc:1467] Processing CreateTablet for tablet adfa098aa30244478834002fa731070e (DEFAULT_TABLE table=test-table [id=10a4a48105614cb395d55c24b097deaf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 04:57:20.180938 12050 ts_tablet_manager.cc:1905] T 364b1d62b33d45379c85dc9c95b68435 P 3c92b56d4ab3443b9c85bdabf4a72977: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 04:57:20.181213 11881 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet adfa098aa30244478834002fa731070e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 04:57:20.181268 11677 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet adfa098aa30244478834002fa731070e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 04:57:20.183099 12047 raft_consensus.cc:2258] T 364b1d62b33d45379c85dc9c95b68435 P bb34b1fe2b6a439fb22a40a0e22d9607 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 04:57:20.187505 12047 ts_tablet_manager.cc:1905] T 364b1d62b33d45379c85dc9c95b68435 P bb34b1fe2b6a439fb22a40a0e22d9607: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 04:57:20.188000 11475 tablet_service.cc:1467] Processing CreateTablet for tablet adfa098aa30244478834002fa731070e (DEFAULT_TABLE table=test-table [id=10a4a48105614cb395d55c24b097deaf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 04:57:20.188303 12050 ts_tablet_manager.cc:1918] T 364b1d62b33d45379c85dc9c95b68435 P 3c92b56d4ab3443b9c85bdabf4a72977: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 04:57:20.188542 12050 log.cc:1196] T 364b1d62b33d45379c85dc9c95b68435 P 3c92b56d4ab3443b9c85bdabf4a72977: Deleting WAL directory at /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785992227050689-9601-0/minicluster-data/ts-1/wal/wals/364b1d62b33d45379c85dc9c95b68435
I20260806 04:57:20.189216 12050 ts_tablet_manager.cc:1939] T 364b1d62b33d45379c85dc9c95b68435 P 3c92b56d4ab3443b9c85bdabf4a72977: Deleting consensus metadata
I20260806 04:57:20.189756 12012 tablet_bootstrap.cc:492] T adfa098aa30244478834002fa731070e P 578ae1f534534b3aa0585dcbd0781ede: Bootstrap starting.
I20260806 04:57:20.191115 11243 catalog_manager.cc:4841] TS 3c92b56d4ab3443b9c85bdabf4a72977 (127.9.96.66:42211): tablet 364b1d62b33d45379c85dc9c95b68435 (table test-table [id=48073d66ee7e45f099e15cf0bdb229a7]) successfully deleted
I20260806 04:57:20.191541 12012 tablet_bootstrap.cc:654] T adfa098aa30244478834002fa731070e P 578ae1f534534b3aa0585dcbd0781ede: Neither blocks nor log segments found. Creating new log.
I20260806 04:57:20.192721 12049 ts_tablet_manager.cc:1905] T 364b1d62b33d45379c85dc9c95b68435 P 578ae1f534534b3aa0585dcbd0781ede: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 04:57:20.193672 11475 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet adfa098aa30244478834002fa731070e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 04:57:20.198992 12012 tablet_bootstrap.cc:492] T adfa098aa30244478834002fa731070e P 578ae1f534534b3aa0585dcbd0781ede: No bootstrap required, opened a new log
I20260806 04:57:20.199224 12012 ts_tablet_manager.cc:1397] T adfa098aa30244478834002fa731070e P 578ae1f534534b3aa0585dcbd0781ede: Time spent bootstrapping tablet: real 0.010s user 0.001s sys 0.004s
I20260806 04:57:20.200206 12012 raft_consensus.cc:348] T adfa098aa30244478834002fa731070e P 578ae1f534534b3aa0585dcbd0781ede [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb34b1fe2b6a439fb22a40a0e22d9607" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 39511 } } peers { permanent_uuid: "3c92b56d4ab3443b9c85bdabf4a72977" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 42211 } } peers { permanent_uuid: "578ae1f534534b3aa0585dcbd0781ede" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 32807 } }
I20260806 04:57:20.200573 12012 raft_consensus.cc:374] T adfa098aa30244478834002fa731070e P 578ae1f534534b3aa0585dcbd0781ede [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 04:57:20.200759 12012 raft_consensus.cc:729] T adfa098aa30244478834002fa731070e P 578ae1f534534b3aa0585dcbd0781ede [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 578ae1f534534b3aa0585dcbd0781ede, State: Initialized, Role: FOLLOWER
I20260806 04:57:20.201112 12012 consensus_queue.cc:260] T adfa098aa30244478834002fa731070e P 578ae1f534534b3aa0585dcbd0781ede [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb34b1fe2b6a439fb22a40a0e22d9607" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 39511 } } peers { permanent_uuid: "3c92b56d4ab3443b9c85bdabf4a72977" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 42211 } } peers { permanent_uuid: "578ae1f534534b3aa0585dcbd0781ede" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 32807 } }
I20260806 04:57:20.201869 12012 ts_tablet_manager.cc:1428] T adfa098aa30244478834002fa731070e P 578ae1f534534b3aa0585dcbd0781ede: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260806 04:57:20.205614 12049 ts_tablet_manager.cc:1918] T 364b1d62b33d45379c85dc9c95b68435 P 578ae1f534534b3aa0585dcbd0781ede: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 04:57:20.205910 12047 ts_tablet_manager.cc:1918] T 364b1d62b33d45379c85dc9c95b68435 P bb34b1fe2b6a439fb22a40a0e22d9607: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 04:57:20.205971 12049 log.cc:1196] T 364b1d62b33d45379c85dc9c95b68435 P 578ae1f534534b3aa0585dcbd0781ede: Deleting WAL directory at /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785992227050689-9601-0/minicluster-data/ts-2/wal/wals/364b1d62b33d45379c85dc9c95b68435
I20260806 04:57:20.206094 12047 log.cc:1196] T 364b1d62b33d45379c85dc9c95b68435 P bb34b1fe2b6a439fb22a40a0e22d9607: Deleting WAL directory at /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785992227050689-9601-0/minicluster-data/ts-0/wal/wals/364b1d62b33d45379c85dc9c95b68435
I20260806 04:57:20.206650 12053 tablet_bootstrap.cc:492] T adfa098aa30244478834002fa731070e P 3c92b56d4ab3443b9c85bdabf4a72977: Bootstrap starting.
I20260806 04:57:20.206697 12047 ts_tablet_manager.cc:1939] T 364b1d62b33d45379c85dc9c95b68435 P bb34b1fe2b6a439fb22a40a0e22d9607: Deleting consensus metadata
I20260806 04:57:20.206791 12049 ts_tablet_manager.cc:1939] T 364b1d62b33d45379c85dc9c95b68435 P 578ae1f534534b3aa0585dcbd0781ede: Deleting consensus metadata
I20260806 04:57:20.208443 12053 tablet_bootstrap.cc:654] T adfa098aa30244478834002fa731070e P 3c92b56d4ab3443b9c85bdabf4a72977: Neither blocks nor log segments found. Creating new log.
I20260806 04:57:20.208510 11239 catalog_manager.cc:4841] TS bb34b1fe2b6a439fb22a40a0e22d9607 (127.9.96.65:39511): tablet 364b1d62b33d45379c85dc9c95b68435 (table test-table [id=48073d66ee7e45f099e15cf0bdb229a7]) successfully deleted
I20260806 04:57:20.208748 11239 catalog_manager.cc:4841] TS 578ae1f534534b3aa0585dcbd0781ede (127.9.96.67:32807): tablet 364b1d62b33d45379c85dc9c95b68435 (table test-table [id=48073d66ee7e45f099e15cf0bdb229a7]) successfully deleted
I20260806 04:57:20.210120 12054 tablet_bootstrap.cc:492] T adfa098aa30244478834002fa731070e P bb34b1fe2b6a439fb22a40a0e22d9607: Bootstrap starting.
I20260806 04:57:20.211555 12054 tablet_bootstrap.cc:654] T adfa098aa30244478834002fa731070e P bb34b1fe2b6a439fb22a40a0e22d9607: Neither blocks nor log segments found. Creating new log.
I20260806 04:57:20.213289 12053 tablet_bootstrap.cc:492] T adfa098aa30244478834002fa731070e P 3c92b56d4ab3443b9c85bdabf4a72977: No bootstrap required, opened a new log
I20260806 04:57:20.213447 12053 ts_tablet_manager.cc:1397] T adfa098aa30244478834002fa731070e P 3c92b56d4ab3443b9c85bdabf4a72977: Time spent bootstrapping tablet: real 0.007s user 0.003s sys 0.000s
I20260806 04:57:20.214066 12053 raft_consensus.cc:348] T adfa098aa30244478834002fa731070e P 3c92b56d4ab3443b9c85bdabf4a72977 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb34b1fe2b6a439fb22a40a0e22d9607" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 39511 } } peers { permanent_uuid: "3c92b56d4ab3443b9c85bdabf4a72977" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 42211 } } peers { permanent_uuid: "578ae1f534534b3aa0585dcbd0781ede" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 32807 } }
I20260806 04:57:20.214244 12053 raft_consensus.cc:374] T adfa098aa30244478834002fa731070e P 3c92b56d4ab3443b9c85bdabf4a72977 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 04:57:20.214391 12053 raft_consensus.cc:729] T adfa098aa30244478834002fa731070e P 3c92b56d4ab3443b9c85bdabf4a72977 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3c92b56d4ab3443b9c85bdabf4a72977, State: Initialized, Role: FOLLOWER
I20260806 04:57:20.214687 12053 consensus_queue.cc:260] T adfa098aa30244478834002fa731070e P 3c92b56d4ab3443b9c85bdabf4a72977 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb34b1fe2b6a439fb22a40a0e22d9607" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 39511 } } peers { permanent_uuid: "3c92b56d4ab3443b9c85bdabf4a72977" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 42211 } } peers { permanent_uuid: "578ae1f534534b3aa0585dcbd0781ede" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 32807 } }
I20260806 04:57:20.215243 12053 ts_tablet_manager.cc:1428] T adfa098aa30244478834002fa731070e P 3c92b56d4ab3443b9c85bdabf4a72977: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260806 04:57:20.218865 12054 tablet_bootstrap.cc:492] T adfa098aa30244478834002fa731070e P bb34b1fe2b6a439fb22a40a0e22d9607: No bootstrap required, opened a new log
I20260806 04:57:20.219017 12054 ts_tablet_manager.cc:1397] T adfa098aa30244478834002fa731070e P bb34b1fe2b6a439fb22a40a0e22d9607: Time spent bootstrapping tablet: real 0.009s user 0.000s sys 0.003s
I20260806 04:57:20.220145 12054 raft_consensus.cc:348] T adfa098aa30244478834002fa731070e P bb34b1fe2b6a439fb22a40a0e22d9607 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb34b1fe2b6a439fb22a40a0e22d9607" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 39511 } } peers { permanent_uuid: "3c92b56d4ab3443b9c85bdabf4a72977" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 42211 } } peers { permanent_uuid: "578ae1f534534b3aa0585dcbd0781ede" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 32807 } }
I20260806 04:57:20.220391 12054 raft_consensus.cc:374] T adfa098aa30244478834002fa731070e P bb34b1fe2b6a439fb22a40a0e22d9607 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 04:57:20.220489 12054 raft_consensus.cc:729] T adfa098aa30244478834002fa731070e P bb34b1fe2b6a439fb22a40a0e22d9607 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bb34b1fe2b6a439fb22a40a0e22d9607, State: Initialized, Role: FOLLOWER
I20260806 04:57:20.220775 12054 consensus_queue.cc:260] T adfa098aa30244478834002fa731070e P bb34b1fe2b6a439fb22a40a0e22d9607 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb34b1fe2b6a439fb22a40a0e22d9607" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 39511 } } peers { permanent_uuid: "3c92b56d4ab3443b9c85bdabf4a72977" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 42211 } } peers { permanent_uuid: "578ae1f534534b3aa0585dcbd0781ede" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 32807 } }
I20260806 04:57:20.221371 12054 ts_tablet_manager.cc:1428] T adfa098aa30244478834002fa731070e P bb34b1fe2b6a439fb22a40a0e22d9607: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260806 04:57:20.288555 12028 raft_consensus.cc:482] T adfa098aa30244478834002fa731070e P 578ae1f534534b3aa0585dcbd0781ede [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 04:57:20.288796 12028 raft_consensus.cc:504] T adfa098aa30244478834002fa731070e P 578ae1f534534b3aa0585dcbd0781ede [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb34b1fe2b6a439fb22a40a0e22d9607" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 39511 } } peers { permanent_uuid: "3c92b56d4ab3443b9c85bdabf4a72977" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 42211 } } peers { permanent_uuid: "578ae1f534534b3aa0585dcbd0781ede" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 32807 } }
I20260806 04:57:20.289651 12028 leader_election.cc:288] T adfa098aa30244478834002fa731070e P 578ae1f534534b3aa0585dcbd0781ede [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bb34b1fe2b6a439fb22a40a0e22d9607 (127.9.96.65:39511), 3c92b56d4ab3443b9c85bdabf4a72977 (127.9.96.66:42211)
I20260806 04:57:20.290184 11718 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "adfa098aa30244478834002fa731070e" candidate_uuid: "578ae1f534534b3aa0585dcbd0781ede" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c92b56d4ab3443b9c85bdabf4a72977" is_pre_election: true
I20260806 04:57:20.290212 11501 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "adfa098aa30244478834002fa731070e" candidate_uuid: "578ae1f534534b3aa0585dcbd0781ede" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb34b1fe2b6a439fb22a40a0e22d9607" is_pre_election: true
I20260806 04:57:20.290464 11718 raft_consensus.cc:2454] T adfa098aa30244478834002fa731070e P 3c92b56d4ab3443b9c85bdabf4a72977 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 578ae1f534534b3aa0585dcbd0781ede in term 0.
I20260806 04:57:20.290468 11501 raft_consensus.cc:2454] T adfa098aa30244478834002fa731070e P bb34b1fe2b6a439fb22a40a0e22d9607 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 578ae1f534534b3aa0585dcbd0781ede in term 0.
I20260806 04:57:20.290944 11816 leader_election.cc:302] T adfa098aa30244478834002fa731070e P 578ae1f534534b3aa0585dcbd0781ede [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 578ae1f534534b3aa0585dcbd0781ede, bb34b1fe2b6a439fb22a40a0e22d9607; no voters:
I20260806 04:57:20.291239 12028 raft_consensus.cc:2781] T adfa098aa30244478834002fa731070e P 578ae1f534534b3aa0585dcbd0781ede [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 04:57:20.291421 12028 raft_consensus.cc:482] T adfa098aa30244478834002fa731070e P 578ae1f534534b3aa0585dcbd0781ede [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 04:57:20.291581 12028 raft_consensus.cc:3037] T adfa098aa30244478834002fa731070e P 578ae1f534534b3aa0585dcbd0781ede [term 0 FOLLOWER]: Advancing to term 1
I20260806 04:57:20.293954 12028 raft_consensus.cc:504] T adfa098aa30244478834002fa731070e P 578ae1f534534b3aa0585dcbd0781ede [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb34b1fe2b6a439fb22a40a0e22d9607" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 39511 } } peers { permanent_uuid: "3c92b56d4ab3443b9c85bdabf4a72977" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 42211 } } peers { permanent_uuid: "578ae1f534534b3aa0585dcbd0781ede" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 32807 } }
I20260806 04:57:20.294549 12028 leader_election.cc:288] T adfa098aa30244478834002fa731070e P 578ae1f534534b3aa0585dcbd0781ede [CANDIDATE]: Term 1 election: Requested vote from peers bb34b1fe2b6a439fb22a40a0e22d9607 (127.9.96.65:39511), 3c92b56d4ab3443b9c85bdabf4a72977 (127.9.96.66:42211)
I20260806 04:57:20.295058 11501 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "adfa098aa30244478834002fa731070e" candidate_uuid: "578ae1f534534b3aa0585dcbd0781ede" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb34b1fe2b6a439fb22a40a0e22d9607"
I20260806 04:57:20.295094 11718 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "adfa098aa30244478834002fa731070e" candidate_uuid: "578ae1f534534b3aa0585dcbd0781ede" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c92b56d4ab3443b9c85bdabf4a72977"
I20260806 04:57:20.295281 11501 raft_consensus.cc:3037] T adfa098aa30244478834002fa731070e P bb34b1fe2b6a439fb22a40a0e22d9607 [term 0 FOLLOWER]: Advancing to term 1
I20260806 04:57:20.295279 11718 raft_consensus.cc:3037] T adfa098aa30244478834002fa731070e P 3c92b56d4ab3443b9c85bdabf4a72977 [term 0 FOLLOWER]: Advancing to term 1
I20260806 04:57:20.298313 11718 raft_consensus.cc:2454] T adfa098aa30244478834002fa731070e P 3c92b56d4ab3443b9c85bdabf4a72977 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 578ae1f534534b3aa0585dcbd0781ede in term 1.
I20260806 04:57:20.298313 11501 raft_consensus.cc:2454] T adfa098aa30244478834002fa731070e P bb34b1fe2b6a439fb22a40a0e22d9607 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 578ae1f534534b3aa0585dcbd0781ede in term 1.
I20260806 04:57:20.298760 11818 leader_election.cc:302] T adfa098aa30244478834002fa731070e P 578ae1f534534b3aa0585dcbd0781ede [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c92b56d4ab3443b9c85bdabf4a72977, 578ae1f534534b3aa0585dcbd0781ede; no voters:
I20260806 04:57:20.299093 12028 raft_consensus.cc:2781] T adfa098aa30244478834002fa731070e P 578ae1f534534b3aa0585dcbd0781ede [term 1 FOLLOWER]: Leader election won for term 1
I20260806 04:57:20.299286 12028 raft_consensus.cc:686] T adfa098aa30244478834002fa731070e P 578ae1f534534b3aa0585dcbd0781ede [term 1 LEADER]: Becoming Leader. State: Replica: 578ae1f534534b3aa0585dcbd0781ede, State: Running, Role: LEADER
I20260806 04:57:20.299686 12028 consensus_queue.cc:237] T adfa098aa30244478834002fa731070e P 578ae1f534534b3aa0585dcbd0781ede [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb34b1fe2b6a439fb22a40a0e22d9607" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 39511 } } peers { permanent_uuid: "3c92b56d4ab3443b9c85bdabf4a72977" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 42211 } } peers { permanent_uuid: "578ae1f534534b3aa0585dcbd0781ede" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 32807 } }
I20260806 04:57:20.302500 11260 catalog_manager.cc:5495] T adfa098aa30244478834002fa731070e P 578ae1f534534b3aa0585dcbd0781ede reported cstate change: term changed from 0 to 1, leader changed from <none> to 578ae1f534534b3aa0585dcbd0781ede (127.9.96.67). New cstate: current_term: 1 leader_uuid: "578ae1f534534b3aa0585dcbd0781ede" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb34b1fe2b6a439fb22a40a0e22d9607" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 39511 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3c92b56d4ab3443b9c85bdabf4a72977" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 42211 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "578ae1f534534b3aa0585dcbd0781ede" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 32807 } health_report { overall_health: HEALTHY } } }
Aug 06 04:57:20 dist-test-slave-x4qh krb5kdc[11195](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992237, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.96.126@KRBTEST.COM for kudu/127.9.96.126@KRBTEST.COM
I20260806 04:57:20.381460 12070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:57:20.365885 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:34519 (local address 127.0.0.1:43170)
0806 04:57:20.366592 (+ 707us) negotiation.cc:107] Waiting for socket to connect
0806 04:57:20.366613 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0806 04:57:20.366789 (+ 176us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:57:20.367240 (+ 451us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:57:20.367245 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:57:20.368148 (+ 903us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:57:20.368464 (+ 316us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:57:20.368476 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:57:20.369701 (+ 1225us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:57:20.369706 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:57:20.371201 (+ 1495us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:57:20.371216 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:57:20.372029 (+ 813us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:57:20.372035 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:57:20.372219 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:57:20.373013 (+ 794us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:57:20.373034 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:57:20.375698 (+ 2664us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 04:57:20.378771 (+ 3073us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:57:20.378786 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:57:20.378793 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:57:20.379187 (+ 394us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:57:20.379611 (+ 424us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:57:20.379616 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:57:20.379620 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:57:20.379786 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:57:20.380315 (+ 529us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 04:57:20.380323 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 04:57:20.380510 (+ 187us) client_negotiation.cc:769] Sending connection context
0806 04:57:20.380675 (+ 165us) client_negotiation.cc:240] Negotiation successful
0806 04:57:20.380979 (+ 304us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":535,"thread_start_us":356,"threads_started":1}
I20260806 04:57:20.381484 12069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:57:20.365882 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43170 (local address 127.9.96.126:34519)
0806 04:57:20.366942 (+ 1060us) server_negotiation.cc:206] Beginning negotiation
0806 04:57:20.366946 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 04:57:20.366966 (+ 20us) server_negotiation.cc:407] Connection header received
0806 04:57:20.367010 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:57:20.367013 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:57:20.367065 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:57:20.367154 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:57:20.368641 (+ 1487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:57:20.369545 (+ 904us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:57:20.371417 (+ 1872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:57:20.371853 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:57:20.371957 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:57:20.375907 (+ 3950us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:57:20.375912 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:57:20.375915 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:57:20.375943 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:57:20.378639 (+ 2696us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:57:20.379310 (+ 671us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:57:20.379317 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:57:20.379323 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:57:20.379504 (+ 181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:57:20.379901 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:57:20.379906 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:57:20.379909 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:57:20.380184 (+ 275us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:57:20.380339 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0806 04:57:20.380751 (+ 412us) server_negotiation.cc:299] Negotiation successful
0806 04:57:20.381098 (+ 347us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":150,"server-negotiator.queue_time_us":955,"thread_start_us":805,"threads_started":1}
I20260806 04:57:20.386760 11260 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.9.96.126@KRBTEST.COM'} at 127.0.0.1:43170:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260806 04:57:20.388018 11260 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 04:57:20.395620 11475 tablet_service.cc:1467] Processing CreateTablet for tablet 7c10ac0104b740a3af175372aa799624 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=427464a3821345fb8b0ba1ac295e48ee]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 04:57:20.396241 11475 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7c10ac0104b740a3af175372aa799624. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 04:57:20.399987 11881 tablet_service.cc:1467] Processing CreateTablet for tablet 7c10ac0104b740a3af175372aa799624 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=427464a3821345fb8b0ba1ac295e48ee]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 04:57:20.400606 11881 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7c10ac0104b740a3af175372aa799624. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 04:57:20.404130 12054 tablet_bootstrap.cc:492] T 7c10ac0104b740a3af175372aa799624 P bb34b1fe2b6a439fb22a40a0e22d9607: Bootstrap starting.
I20260806 04:57:20.404172 11677 tablet_service.cc:1467] Processing CreateTablet for tablet 7c10ac0104b740a3af175372aa799624 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=427464a3821345fb8b0ba1ac295e48ee]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 04:57:20.404799 11677 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7c10ac0104b740a3af175372aa799624. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 04:57:20.405979 12054 tablet_bootstrap.cc:654] T 7c10ac0104b740a3af175372aa799624 P bb34b1fe2b6a439fb22a40a0e22d9607: Neither blocks nor log segments found. Creating new log.
I20260806 04:57:20.408501 12012 tablet_bootstrap.cc:492] T 7c10ac0104b740a3af175372aa799624 P 578ae1f534534b3aa0585dcbd0781ede: Bootstrap starting.
I20260806 04:57:20.410321 12012 tablet_bootstrap.cc:654] T 7c10ac0104b740a3af175372aa799624 P 578ae1f534534b3aa0585dcbd0781ede: Neither blocks nor log segments found. Creating new log.
I20260806 04:57:20.413035 12053 tablet_bootstrap.cc:492] T 7c10ac0104b740a3af175372aa799624 P 3c92b56d4ab3443b9c85bdabf4a72977: Bootstrap starting.
I20260806 04:57:20.415055 12053 tablet_bootstrap.cc:654] T 7c10ac0104b740a3af175372aa799624 P 3c92b56d4ab3443b9c85bdabf4a72977: Neither blocks nor log segments found. Creating new log.
I20260806 04:57:20.421649 12054 tablet_bootstrap.cc:492] T 7c10ac0104b740a3af175372aa799624 P bb34b1fe2b6a439fb22a40a0e22d9607: No bootstrap required, opened a new log
I20260806 04:57:20.421895 12054 ts_tablet_manager.cc:1397] T 7c10ac0104b740a3af175372aa799624 P bb34b1fe2b6a439fb22a40a0e22d9607: Time spent bootstrapping tablet: real 0.018s user 0.001s sys 0.012s
I20260806 04:57:20.422940 12054 raft_consensus.cc:348] T 7c10ac0104b740a3af175372aa799624 P bb34b1fe2b6a439fb22a40a0e22d9607 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb34b1fe2b6a439fb22a40a0e22d9607" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 39511 } } peers { permanent_uuid: "3c92b56d4ab3443b9c85bdabf4a72977" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 42211 } } peers { permanent_uuid: "578ae1f534534b3aa0585dcbd0781ede" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 32807 } }
I20260806 04:57:20.423418 12054 raft_consensus.cc:374] T 7c10ac0104b740a3af175372aa799624 P bb34b1fe2b6a439fb22a40a0e22d9607 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 04:57:20.423578 12054 raft_consensus.cc:729] T 7c10ac0104b740a3af175372aa799624 P bb34b1fe2b6a439fb22a40a0e22d9607 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bb34b1fe2b6a439fb22a40a0e22d9607, State: Initialized, Role: FOLLOWER
I20260806 04:57:20.424253 12054 consensus_queue.cc:260] T 7c10ac0104b740a3af175372aa799624 P bb34b1fe2b6a439fb22a40a0e22d9607 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb34b1fe2b6a439fb22a40a0e22d9607" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 39511 } } peers { permanent_uuid: "3c92b56d4ab3443b9c85bdabf4a72977" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 42211 } } peers { permanent_uuid: "578ae1f534534b3aa0585dcbd0781ede" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 32807 } }
I20260806 04:57:20.424782 12012 tablet_bootstrap.cc:492] T 7c10ac0104b740a3af175372aa799624 P 578ae1f534534b3aa0585dcbd0781ede: No bootstrap required, opened a new log
I20260806 04:57:20.424953 12012 ts_tablet_manager.cc:1397] T 7c10ac0104b740a3af175372aa799624 P 578ae1f534534b3aa0585dcbd0781ede: Time spent bootstrapping tablet: real 0.017s user 0.011s sys 0.004s
I20260806 04:57:20.425159 12054 ts_tablet_manager.cc:1428] T 7c10ac0104b740a3af175372aa799624 P bb34b1fe2b6a439fb22a40a0e22d9607: Time spent starting tablet: real 0.003s user 0.000s sys 0.002s
I20260806 04:57:20.424918 12024 tablet_replica.cc:439] T 7c10ac0104b740a3af175372aa799624 P bb34b1fe2b6a439fb22a40a0e22d9607: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb34b1fe2b6a439fb22a40a0e22d9607" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 39511 } } peers { permanent_uuid: "3c92b56d4ab3443b9c85bdabf4a72977" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 42211 } } peers { permanent_uuid: "578ae1f534534b3aa0585dcbd0781ede" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 32807 } } }
I20260806 04:57:20.425354 12024 tablet_replica.cc:442] T 7c10ac0104b740a3af175372aa799624 P bb34b1fe2b6a439fb22a40a0e22d9607: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 04:57:20.425740 12012 raft_consensus.cc:348] T 7c10ac0104b740a3af175372aa799624 P 578ae1f534534b3aa0585dcbd0781ede [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb34b1fe2b6a439fb22a40a0e22d9607" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 39511 } } peers { permanent_uuid: "3c92b56d4ab3443b9c85bdabf4a72977" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 42211 } } peers { permanent_uuid: "578ae1f534534b3aa0585dcbd0781ede" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 32807 } }
I20260806 04:57:20.426064 12012 raft_consensus.cc:374] T 7c10ac0104b740a3af175372aa799624 P 578ae1f534534b3aa0585dcbd0781ede [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 04:57:20.426165 12012 raft_consensus.cc:729] T 7c10ac0104b740a3af175372aa799624 P 578ae1f534534b3aa0585dcbd0781ede [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 578ae1f534534b3aa0585dcbd0781ede, State: Initialized, Role: FOLLOWER
I20260806 04:57:20.426429 12012 consensus_queue.cc:260] T 7c10ac0104b740a3af175372aa799624 P 578ae1f534534b3aa0585dcbd0781ede [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb34b1fe2b6a439fb22a40a0e22d9607" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 39511 } } peers { permanent_uuid: "3c92b56d4ab3443b9c85bdabf4a72977" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 42211 } } peers { permanent_uuid: "578ae1f534534b3aa0585dcbd0781ede" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 32807 } }
I20260806 04:57:20.426878 12028 tablet_replica.cc:439] T 7c10ac0104b740a3af175372aa799624 P 578ae1f534534b3aa0585dcbd0781ede: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb34b1fe2b6a439fb22a40a0e22d9607" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 39511 } } peers { permanent_uuid: "3c92b56d4ab3443b9c85bdabf4a72977" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 42211 } } peers { permanent_uuid: "578ae1f534534b3aa0585dcbd0781ede" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 32807 } } }
I20260806 04:57:20.427168 12028 tablet_replica.cc:442] T 7c10ac0104b740a3af175372aa799624 P 578ae1f534534b3aa0585dcbd0781ede: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 04:57:20.427660 12012 ts_tablet_manager.cc:1428] T 7c10ac0104b740a3af175372aa799624 P 578ae1f534534b3aa0585dcbd0781ede: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260806 04:57:20.429622 12053 tablet_bootstrap.cc:492] T 7c10ac0104b740a3af175372aa799624 P 3c92b56d4ab3443b9c85bdabf4a72977: No bootstrap required, opened a new log
I20260806 04:57:20.429829 12053 ts_tablet_manager.cc:1397] T 7c10ac0104b740a3af175372aa799624 P 3c92b56d4ab3443b9c85bdabf4a72977: Time spent bootstrapping tablet: real 0.017s user 0.007s sys 0.006s
I20260806 04:57:20.430661 12053 raft_consensus.cc:348] T 7c10ac0104b740a3af175372aa799624 P 3c92b56d4ab3443b9c85bdabf4a72977 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb34b1fe2b6a439fb22a40a0e22d9607" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 39511 } } peers { permanent_uuid: "3c92b56d4ab3443b9c85bdabf4a72977" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 42211 } } peers { permanent_uuid: "578ae1f534534b3aa0585dcbd0781ede" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 32807 } }
I20260806 04:57:20.430972 12053 raft_consensus.cc:374] T 7c10ac0104b740a3af175372aa799624 P 3c92b56d4ab3443b9c85bdabf4a72977 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 04:57:20.431071 12053 raft_consensus.cc:729] T 7c10ac0104b740a3af175372aa799624 P 3c92b56d4ab3443b9c85bdabf4a72977 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3c92b56d4ab3443b9c85bdabf4a72977, State: Initialized, Role: FOLLOWER
I20260806 04:57:20.431411 12053 consensus_queue.cc:260] T 7c10ac0104b740a3af175372aa799624 P 3c92b56d4ab3443b9c85bdabf4a72977 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb34b1fe2b6a439fb22a40a0e22d9607" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 39511 } } peers { permanent_uuid: "3c92b56d4ab3443b9c85bdabf4a72977" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 42211 } } peers { permanent_uuid: "578ae1f534534b3aa0585dcbd0781ede" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 32807 } }
I20260806 04:57:20.432019 12053 ts_tablet_manager.cc:1428] T 7c10ac0104b740a3af175372aa799624 P 3c92b56d4ab3443b9c85bdabf4a72977: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20260806 04:57:20.432407 12023 tablet_replica.cc:439] T 7c10ac0104b740a3af175372aa799624 P 3c92b56d4ab3443b9c85bdabf4a72977: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb34b1fe2b6a439fb22a40a0e22d9607" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 39511 } } peers { permanent_uuid: "3c92b56d4ab3443b9c85bdabf4a72977" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 42211 } } peers { permanent_uuid: "578ae1f534534b3aa0585dcbd0781ede" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 32807 } } }
I20260806 04:57:20.432819 12023 tablet_replica.cc:442] T 7c10ac0104b740a3af175372aa799624 P 3c92b56d4ab3443b9c85bdabf4a72977: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 04:57:20.513516 12024 raft_consensus.cc:482] T 7c10ac0104b740a3af175372aa799624 P bb34b1fe2b6a439fb22a40a0e22d9607 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 04:57:20.513720 12024 raft_consensus.cc:504] T 7c10ac0104b740a3af175372aa799624 P bb34b1fe2b6a439fb22a40a0e22d9607 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb34b1fe2b6a439fb22a40a0e22d9607" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 39511 } } peers { permanent_uuid: "3c92b56d4ab3443b9c85bdabf4a72977" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 42211 } } peers { permanent_uuid: "578ae1f534534b3aa0585dcbd0781ede" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 32807 } }
I20260806 04:57:20.515319 12024 leader_election.cc:288] T 7c10ac0104b740a3af175372aa799624 P bb34b1fe2b6a439fb22a40a0e22d9607 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3c92b56d4ab3443b9c85bdabf4a72977 (127.9.96.66:42211), 578ae1f534534b3aa0585dcbd0781ede (127.9.96.67:32807)
I20260806 04:57:20.535313 12004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:57:20.516197 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.96.65:34237 (local address 127.9.96.67:32807)
0806 04:57:20.516410 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0806 04:57:20.516417 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 04:57:20.527002 (+ 10585us) server_negotiation.cc:407] Connection header received
0806 04:57:20.527680 (+ 678us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:57:20.527687 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:57:20.527778 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:57:20.527882 (+ 104us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 04:57:20.528726 (+ 844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:57:20.529933 (+ 1207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:57:20.532622 (+ 2689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:57:20.534090 (+ 1468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:57:20.534190 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:57:20.534289 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0806 04:57:20.534855 (+ 566us) server_negotiation.cc:299] Negotiation successful
0806 04:57:20.535026 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260806 04:57:20.536338 12079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:57:20.515718 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.67:32807 (local address 127.9.96.65:34237)
0806 04:57:20.526884 (+ 11166us) negotiation.cc:107] Waiting for socket to connect
0806 04:57:20.526909 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0806 04:57:20.527536 (+ 627us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:57:20.528084 (+ 548us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:57:20.528090 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:57:20.528116 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 04:57:20.528515 (+ 399us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:57:20.528528 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:57:20.530145 (+ 1617us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:57:20.530150 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:57:20.532405 (+ 2255us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:57:20.532421 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:57:20.534431 (+ 2010us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:57:20.534437 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:57:20.534638 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:57:20.534674 (+ 36us) client_negotiation.cc:769] Sending connection context
0806 04:57:20.535809 (+ 1135us) client_negotiation.cc:240] Negotiation successful
0806 04:57:20.535953 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11040,"thread_start_us":10932,"threads_started":1}
I20260806 04:57:20.537623 11917 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c10ac0104b740a3af175372aa799624" candidate_uuid: "bb34b1fe2b6a439fb22a40a0e22d9607" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "578ae1f534534b3aa0585dcbd0781ede" is_pre_election: true
I20260806 04:57:20.538033 11917 raft_consensus.cc:2454] T 7c10ac0104b740a3af175372aa799624 P 578ae1f534534b3aa0585dcbd0781ede [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bb34b1fe2b6a439fb22a40a0e22d9607 in term 0.
I20260806 04:57:20.538722 11395 leader_election.cc:302] T 7c10ac0104b740a3af175372aa799624 P bb34b1fe2b6a439fb22a40a0e22d9607 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 578ae1f534534b3aa0585dcbd0781ede, bb34b1fe2b6a439fb22a40a0e22d9607; no voters:
I20260806 04:57:20.539150 12024 raft_consensus.cc:2781] T 7c10ac0104b740a3af175372aa799624 P bb34b1fe2b6a439fb22a40a0e22d9607 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 04:57:20.539269 12024 raft_consensus.cc:482] T 7c10ac0104b740a3af175372aa799624 P bb34b1fe2b6a439fb22a40a0e22d9607 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 04:57:20.539409 12024 raft_consensus.cc:3037] T 7c10ac0104b740a3af175372aa799624 P bb34b1fe2b6a439fb22a40a0e22d9607 [term 0 FOLLOWER]: Advancing to term 1
I20260806 04:57:20.542791 12024 raft_consensus.cc:504] T 7c10ac0104b740a3af175372aa799624 P bb34b1fe2b6a439fb22a40a0e22d9607 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb34b1fe2b6a439fb22a40a0e22d9607" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 39511 } } peers { permanent_uuid: "3c92b56d4ab3443b9c85bdabf4a72977" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 42211 } } peers { permanent_uuid: "578ae1f534534b3aa0585dcbd0781ede" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 32807 } }
I20260806 04:57:20.544059 11917 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c10ac0104b740a3af175372aa799624" candidate_uuid: "bb34b1fe2b6a439fb22a40a0e22d9607" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "578ae1f534534b3aa0585dcbd0781ede"
I20260806 04:57:20.544266 11917 raft_consensus.cc:3037] T 7c10ac0104b740a3af175372aa799624 P 578ae1f534534b3aa0585dcbd0781ede [term 0 FOLLOWER]: Advancing to term 1
I20260806 04:57:20.545106 12080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:57:20.515745 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.66:42211 (local address 127.9.96.65:37837)
0806 04:57:20.527290 (+ 11545us) negotiation.cc:107] Waiting for socket to connect
0806 04:57:20.527309 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0806 04:57:20.527467 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:57:20.538027 (+ 10560us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:57:20.538032 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:57:20.538066 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 04:57:20.538466 (+ 400us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:57:20.538478 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:57:20.540227 (+ 1749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:57:20.540232 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:57:20.542514 (+ 2282us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:57:20.542530 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:57:20.544264 (+ 1734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:57:20.544272 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:57:20.544488 (+ 216us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:57:20.544533 (+ 45us) client_negotiation.cc:769] Sending connection context
0806 04:57:20.544608 (+ 75us) client_negotiation.cc:240] Negotiation successful
0806 04:57:20.544730 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11411,"thread_start_us":11636,"threads_started":1}
I20260806 04:57:20.546021 12077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:57:20.515682 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.96.65:37837 (local address 127.9.96.66:42211)
0806 04:57:20.537632 (+ 21950us) server_negotiation.cc:206] Beginning negotiation
0806 04:57:20.537638 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 04:57:20.537659 (+ 21us) server_negotiation.cc:407] Connection header received
0806 04:57:20.537730 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:57:20.537736 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:57:20.537853 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:57:20.538697 (+ 844us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 04:57:20.538893 (+ 196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:57:20.540042 (+ 1149us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:57:20.542732 (+ 2690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:57:20.544019 (+ 1287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:57:20.545356 (+ 1337us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:57:20.545461 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0806 04:57:20.545518 (+ 57us) server_negotiation.cc:299] Negotiation successful
0806 04:57:20.545665 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21810,"thread_start_us":21690,"threads_started":1}
I20260806 04:57:20.546967 11718 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c10ac0104b740a3af175372aa799624" candidate_uuid: "bb34b1fe2b6a439fb22a40a0e22d9607" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c92b56d4ab3443b9c85bdabf4a72977" is_pre_election: true
I20260806 04:57:20.547068 12024 leader_election.cc:288] T 7c10ac0104b740a3af175372aa799624 P bb34b1fe2b6a439fb22a40a0e22d9607 [CANDIDATE]: Term 1 election: Requested vote from peers 3c92b56d4ab3443b9c85bdabf4a72977 (127.9.96.66:42211), 578ae1f534534b3aa0585dcbd0781ede (127.9.96.67:32807)
I20260806 04:57:20.547235 11718 raft_consensus.cc:2454] T 7c10ac0104b740a3af175372aa799624 P 3c92b56d4ab3443b9c85bdabf4a72977 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bb34b1fe2b6a439fb22a40a0e22d9607 in term 0.
I20260806 04:57:20.547210 11713 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c10ac0104b740a3af175372aa799624" candidate_uuid: "bb34b1fe2b6a439fb22a40a0e22d9607" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c92b56d4ab3443b9c85bdabf4a72977"
I20260806 04:57:20.547590 11713 raft_consensus.cc:3037] T 7c10ac0104b740a3af175372aa799624 P 3c92b56d4ab3443b9c85bdabf4a72977 [term 0 FOLLOWER]: Advancing to term 1
I20260806 04:57:20.547581 11917 raft_consensus.cc:2454] T 7c10ac0104b740a3af175372aa799624 P 578ae1f534534b3aa0585dcbd0781ede [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bb34b1fe2b6a439fb22a40a0e22d9607 in term 1.
I20260806 04:57:20.547996 11395 leader_election.cc:302] T 7c10ac0104b740a3af175372aa799624 P bb34b1fe2b6a439fb22a40a0e22d9607 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 578ae1f534534b3aa0585dcbd0781ede, bb34b1fe2b6a439fb22a40a0e22d9607; no voters:
I20260806 04:57:20.548308 12024 raft_consensus.cc:2781] T 7c10ac0104b740a3af175372aa799624 P bb34b1fe2b6a439fb22a40a0e22d9607 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 04:57:20.550717 11713 raft_consensus.cc:2454] T 7c10ac0104b740a3af175372aa799624 P 3c92b56d4ab3443b9c85bdabf4a72977 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bb34b1fe2b6a439fb22a40a0e22d9607 in term 1.
I20260806 04:57:20.551637 12024 raft_consensus.cc:686] T 7c10ac0104b740a3af175372aa799624 P bb34b1fe2b6a439fb22a40a0e22d9607 [term 1 LEADER]: Becoming Leader. State: Replica: bb34b1fe2b6a439fb22a40a0e22d9607, State: Running, Role: LEADER
I20260806 04:57:20.552392 12024 consensus_queue.cc:237] T 7c10ac0104b740a3af175372aa799624 P bb34b1fe2b6a439fb22a40a0e22d9607 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb34b1fe2b6a439fb22a40a0e22d9607" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 39511 } } peers { permanent_uuid: "3c92b56d4ab3443b9c85bdabf4a72977" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 42211 } } peers { permanent_uuid: "578ae1f534534b3aa0585dcbd0781ede" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 32807 } }
I20260806 04:57:20.563292 12081 tablet_replica.cc:439] T 7c10ac0104b740a3af175372aa799624 P bb34b1fe2b6a439fb22a40a0e22d9607: TxnStatusTablet state changed. Reason: New leader bb34b1fe2b6a439fb22a40a0e22d9607. Latest consensus state: current_term: 1 leader_uuid: "bb34b1fe2b6a439fb22a40a0e22d9607" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb34b1fe2b6a439fb22a40a0e22d9607" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 39511 } } peers { permanent_uuid: "3c92b56d4ab3443b9c85bdabf4a72977" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 42211 } } peers { permanent_uuid: "578ae1f534534b3aa0585dcbd0781ede" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 32807 } } }
I20260806 04:57:20.563660 12081 tablet_replica.cc:442] T 7c10ac0104b740a3af175372aa799624 P bb34b1fe2b6a439fb22a40a0e22d9607: This TxnStatusTablet replica's current role is: LEADER
I20260806 04:57:20.563494 11260 catalog_manager.cc:5495] T 7c10ac0104b740a3af175372aa799624 P bb34b1fe2b6a439fb22a40a0e22d9607 reported cstate change: term changed from 0 to 1, leader changed from <none> to bb34b1fe2b6a439fb22a40a0e22d9607 (127.9.96.65). New cstate: current_term: 1 leader_uuid: "bb34b1fe2b6a439fb22a40a0e22d9607" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb34b1fe2b6a439fb22a40a0e22d9607" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 39511 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3c92b56d4ab3443b9c85bdabf4a72977" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 42211 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "578ae1f534534b3aa0585dcbd0781ede" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 32807 } health_report { overall_health: UNKNOWN } } }
I20260806 04:57:20.569036 12083 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260806 04:57:20.569353 12083 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260806 04:57:20.611042 12070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:57:20.601242 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.65:39511 (local address 127.0.0.1:34354)
0806 04:57:20.601512 (+ 270us) negotiation.cc:107] Waiting for socket to connect
0806 04:57:20.601534 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0806 04:57:20.601673 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:57:20.604351 (+ 2678us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:57:20.604357 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:57:20.604395 (+ 38us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 04:57:20.604617 (+ 222us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:57:20.604630 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:57:20.606322 (+ 1692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:57:20.606327 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:57:20.607955 (+ 1628us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:57:20.607971 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:57:20.609018 (+ 1047us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:57:20.609025 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:57:20.609229 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:57:20.609312 (+ 83us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 04:57:20.610439 (+ 1127us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 04:57:20.610462 (+ 23us) client_negotiation.cc:769] Sending connection context
0806 04:57:20.610559 (+ 97us) client_negotiation.cc:240] Negotiation successful
0806 04:57:20.610717 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":60}
I20260806 04:57:20.611126 12092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:57:20.601628 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34354 (local address 127.9.96.65:39511)
0806 04:57:20.603891 (+ 2263us) server_negotiation.cc:206] Beginning negotiation
0806 04:57:20.603896 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 04:57:20.603920 (+ 24us) server_negotiation.cc:407] Connection header received
0806 04:57:20.604025 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:57:20.604032 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:57:20.604134 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:57:20.604309 (+ 175us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 04:57:20.604799 (+ 490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:57:20.606175 (+ 1376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:57:20.608114 (+ 1939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:57:20.608840 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:57:20.608984 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:57:20.609572 (+ 588us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 04:57:20.610304 (+ 732us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 04:57:20.610449 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0806 04:57:20.610620 (+ 171us) server_negotiation.cc:299] Negotiation successful
0806 04:57:20.610711 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2117,"thread_start_us":1937,"threads_started":1}
I20260806 04:57:20.616278 11713 raft_consensus.cc:1261] T 7c10ac0104b740a3af175372aa799624 P 3c92b56d4ab3443b9c85bdabf4a72977 [term 1 FOLLOWER]: Refusing update from remote peer bb34b1fe2b6a439fb22a40a0e22d9607: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 04:57:20.616410 11917 raft_consensus.cc:1261] T 7c10ac0104b740a3af175372aa799624 P 578ae1f534534b3aa0585dcbd0781ede [term 1 FOLLOWER]: Refusing update from remote peer bb34b1fe2b6a439fb22a40a0e22d9607: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 04:57:20.617064 12024 consensus_queue.cc:1035] T 7c10ac0104b740a3af175372aa799624 P bb34b1fe2b6a439fb22a40a0e22d9607 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c92b56d4ab3443b9c85bdabf4a72977" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 42211 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 04:57:20.617404 12081 consensus_queue.cc:1035] T 7c10ac0104b740a3af175372aa799624 P bb34b1fe2b6a439fb22a40a0e22d9607 [LEADER]: Connected to new peer: Peer: permanent_uuid: "578ae1f534534b3aa0585dcbd0781ede" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 32807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 04:57:20.632946 12023 tablet_replica.cc:439] T 7c10ac0104b740a3af175372aa799624 P 3c92b56d4ab3443b9c85bdabf4a72977: TxnStatusTablet state changed. Reason: New leader bb34b1fe2b6a439fb22a40a0e22d9607. Latest consensus state: current_term: 1 leader_uuid: "bb34b1fe2b6a439fb22a40a0e22d9607" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb34b1fe2b6a439fb22a40a0e22d9607" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 39511 } } peers { permanent_uuid: "3c92b56d4ab3443b9c85bdabf4a72977" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 42211 } } peers { permanent_uuid: "578ae1f534534b3aa0585dcbd0781ede" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 32807 } } }
I20260806 04:57:20.633268 12023 tablet_replica.cc:442] T 7c10ac0104b740a3af175372aa799624 P 3c92b56d4ab3443b9c85bdabf4a72977: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 04:57:20.637523 12024 tablet_replica.cc:439] T 7c10ac0104b740a3af175372aa799624 P bb34b1fe2b6a439fb22a40a0e22d9607: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "bb34b1fe2b6a439fb22a40a0e22d9607" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb34b1fe2b6a439fb22a40a0e22d9607" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 39511 } } peers { permanent_uuid: "3c92b56d4ab3443b9c85bdabf4a72977" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 42211 } } peers { permanent_uuid: "578ae1f534534b3aa0585dcbd0781ede" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 32807 } } }
I20260806 04:57:20.637835 12024 tablet_replica.cc:442] T 7c10ac0104b740a3af175372aa799624 P bb34b1fe2b6a439fb22a40a0e22d9607: This TxnStatusTablet replica's current role is: LEADER
I20260806 04:57:20.641705 12028 tablet_replica.cc:439] T 7c10ac0104b740a3af175372aa799624 P 578ae1f534534b3aa0585dcbd0781ede: TxnStatusTablet state changed. Reason: New leader bb34b1fe2b6a439fb22a40a0e22d9607. Latest consensus state: current_term: 1 leader_uuid: "bb34b1fe2b6a439fb22a40a0e22d9607" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb34b1fe2b6a439fb22a40a0e22d9607" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 39511 } } peers { permanent_uuid: "3c92b56d4ab3443b9c85bdabf4a72977" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 42211 } } peers { permanent_uuid: "578ae1f534534b3aa0585dcbd0781ede" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 32807 } } }
I20260806 04:57:20.642037 12028 tablet_replica.cc:442] T 7c10ac0104b740a3af175372aa799624 P 578ae1f534534b3aa0585dcbd0781ede: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 04:57:20.642381 12081 tablet_replica.cc:439] T 7c10ac0104b740a3af175372aa799624 P bb34b1fe2b6a439fb22a40a0e22d9607: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "bb34b1fe2b6a439fb22a40a0e22d9607" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb34b1fe2b6a439fb22a40a0e22d9607" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 39511 } } peers { permanent_uuid: "3c92b56d4ab3443b9c85bdabf4a72977" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 42211 } } peers { permanent_uuid: "578ae1f534534b3aa0585dcbd0781ede" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 32807 } } }
I20260806 04:57:20.642616 12081 tablet_replica.cc:442] T 7c10ac0104b740a3af175372aa799624 P bb34b1fe2b6a439fb22a40a0e22d9607: This TxnStatusTablet replica's current role is: LEADER
I20260806 04:57:20.647542 12033 mvcc.cc:205] Tried to move back new op lower bound from 7315424217555472384 to 7315424217306865664. Current Snapshot: MvccSnapshot[applied={T|T < 7315424217555472384}]
I20260806 04:57:20.679669 12023 tablet_replica.cc:439] T 7c10ac0104b740a3af175372aa799624 P 3c92b56d4ab3443b9c85bdabf4a72977: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "bb34b1fe2b6a439fb22a40a0e22d9607" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb34b1fe2b6a439fb22a40a0e22d9607" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 39511 } } peers { permanent_uuid: "3c92b56d4ab3443b9c85bdabf4a72977" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 42211 } } peers { permanent_uuid: "578ae1f534534b3aa0585dcbd0781ede" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 32807 } } }
I20260806 04:57:20.679947 12023 tablet_replica.cc:442] T 7c10ac0104b740a3af175372aa799624 P 3c92b56d4ab3443b9c85bdabf4a72977: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 04:57:20.684747 12028 tablet_replica.cc:439] T 7c10ac0104b740a3af175372aa799624 P 578ae1f534534b3aa0585dcbd0781ede: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "bb34b1fe2b6a439fb22a40a0e22d9607" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb34b1fe2b6a439fb22a40a0e22d9607" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 39511 } } peers { permanent_uuid: "3c92b56d4ab3443b9c85bdabf4a72977" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 42211 } } peers { permanent_uuid: "578ae1f534534b3aa0585dcbd0781ede" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 32807 } } }
I20260806 04:57:20.685072 12028 tablet_replica.cc:442] T 7c10ac0104b740a3af175372aa799624 P 578ae1f534534b3aa0585dcbd0781ede: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 04:57:20.694008 12106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:57:20.688343 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.65:39511 (local address 127.9.96.67:38337)
0806 04:57:20.688956 (+ 613us) negotiation.cc:107] Waiting for socket to connect
0806 04:57:20.688972 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 04:57:20.689146 (+ 174us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:57:20.689497 (+ 351us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:57:20.689501 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:57:20.689514 (+ 13us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 04:57:20.689824 (+ 310us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:57:20.689834 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:57:20.690949 (+ 1115us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:57:20.690956 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:57:20.692057 (+ 1101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:57:20.692069 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:57:20.692822 (+ 753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:57:20.692829 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:57:20.692980 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:57:20.693033 (+ 53us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 04:57:20.693414 (+ 381us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 04:57:20.693428 (+ 14us) client_negotiation.cc:769] Sending connection context
0806 04:57:20.693544 (+ 116us) client_negotiation.cc:240] Negotiation successful
0806 04:57:20.693716 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":469,"thread_start_us":312,"threads_started":1}
I20260806 04:57:20.694049 12092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:57:20.688358 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.96.67:38337 (local address 127.9.96.65:39511)
0806 04:57:20.688516 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0806 04:57:20.688522 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 04:57:20.689068 (+ 546us) server_negotiation.cc:407] Connection header received
0806 04:57:20.689311 (+ 243us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:57:20.689315 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:57:20.689382 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:57:20.689481 (+ 99us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 04:57:20.689991 (+ 510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:57:20.690797 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:57:20.692201 (+ 1404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:57:20.692630 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:57:20.692788 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:57:20.693208 (+ 420us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 04:57:20.693306 (+ 98us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 04:57:20.693414 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0806 04:57:20.693620 (+ 206us) server_negotiation.cc:299] Negotiation successful
0806 04:57:20.693731 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260806 04:57:20.704104 11713 raft_consensus.cc:1261] T adfa098aa30244478834002fa731070e P 3c92b56d4ab3443b9c85bdabf4a72977 [term 1 FOLLOWER]: Refusing update from remote peer 578ae1f534534b3aa0585dcbd0781ede: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 04:57:20.704202 11501 raft_consensus.cc:1261] T adfa098aa30244478834002fa731070e P bb34b1fe2b6a439fb22a40a0e22d9607 [term 1 FOLLOWER]: Refusing update from remote peer 578ae1f534534b3aa0585dcbd0781ede: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 04:57:20.704794 12040 consensus_queue.cc:1035] T adfa098aa30244478834002fa731070e P 578ae1f534534b3aa0585dcbd0781ede [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c92b56d4ab3443b9c85bdabf4a72977" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 42211 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 04:57:20.705104 12028 consensus_queue.cc:1035] T adfa098aa30244478834002fa731070e P 578ae1f534534b3aa0585dcbd0781ede [LEADER]: Connected to new peer: Peer: permanent_uuid: "bb34b1fe2b6a439fb22a40a0e22d9607" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 39511 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 04:57:20.746676 12004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:57:20.739623 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.96.65:46853 (local address 127.9.96.67:32807)
0806 04:57:20.739817 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0806 04:57:20.739824 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 04:57:20.740576 (+ 752us) server_negotiation.cc:407] Connection header received
0806 04:57:20.740864 (+ 288us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:57:20.740870 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:57:20.740948 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:57:20.741051 (+ 103us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 04:57:20.741636 (+ 585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:57:20.742529 (+ 893us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:57:20.744798 (+ 2269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:57:20.745254 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:57:20.745403 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:57:20.745920 (+ 517us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 04:57:20.746029 (+ 109us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 04:57:20.746137 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0806 04:57:20.746293 (+ 156us) server_negotiation.cc:299] Negotiation successful
0806 04:57:20.746374 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260806 04:57:20.747000 12113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:57:20.739644 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.67:32807 (local address 127.9.96.65:46853)
0806 04:57:20.740454 (+ 810us) negotiation.cc:107] Waiting for socket to connect
0806 04:57:20.740476 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0806 04:57:20.740670 (+ 194us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:57:20.741069 (+ 399us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:57:20.741074 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:57:20.741087 (+ 13us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 04:57:20.741472 (+ 385us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:57:20.741482 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:57:20.742662 (+ 1180us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:57:20.742669 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:57:20.744639 (+ 1970us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:57:20.744653 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:57:20.745477 (+ 824us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:57:20.745484 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:57:20.745635 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:57:20.745687 (+ 52us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 04:57:20.746140 (+ 453us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 04:57:20.746156 (+ 16us) client_negotiation.cc:769] Sending connection context
0806 04:57:20.746267 (+ 111us) client_negotiation.cc:240] Negotiation successful
0806 04:57:20.746419 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":599,"thread_start_us":383,"threads_started":1}
I20260806 04:57:20.781711 11264 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:51904:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 04:57:20.781942 11264 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:51904:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 04:57:20.785203 11264 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d1b98044e21d4333a884f6762082b055: Sending DeleteTablet for 3 replicas of tablet adfa098aa30244478834002fa731070e
I20260806 04:57:20.786216 11677 tablet_service.cc:1514] Processing DeleteTablet for tablet adfa098aa30244478834002fa731070e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 04:57:20 UTC) from {username='kudu', principal='kudu/127.9.96.126@KRBTEST.COM'} at 127.0.0.1:33626
I20260806 04:57:20.786345 11881 tablet_service.cc:1514] Processing DeleteTablet for tablet adfa098aa30244478834002fa731070e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 04:57:20 UTC) from {username='kudu', principal='kudu/127.9.96.126@KRBTEST.COM'} at 127.0.0.1:33250
I20260806 04:57:20.786703 11475 tablet_service.cc:1514] Processing DeleteTablet for tablet adfa098aa30244478834002fa731070e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 04:57:20 UTC) from {username='kudu', principal='kudu/127.9.96.126@KRBTEST.COM'} at 127.0.0.1:40950
I20260806 04:57:20.795650 12119 tablet_replica.cc:330] T adfa098aa30244478834002fa731070e P bb34b1fe2b6a439fb22a40a0e22d9607: stopping tablet replica
I20260806 04:57:20.795986 12119 raft_consensus.cc:2229] T adfa098aa30244478834002fa731070e P bb34b1fe2b6a439fb22a40a0e22d9607 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 04:57:20.796438 12119 raft_consensus.cc:2258] T adfa098aa30244478834002fa731070e P bb34b1fe2b6a439fb22a40a0e22d9607 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 04:57:20.798502 12118 tablet_replica.cc:330] T adfa098aa30244478834002fa731070e P 578ae1f534534b3aa0585dcbd0781ede: stopping tablet replica
I20260806 04:57:20.798895 12118 raft_consensus.cc:2229] T adfa098aa30244478834002fa731070e P 578ae1f534534b3aa0585dcbd0781ede [term 1 LEADER]: Raft consensus shutting down.
I20260806 04:57:20.799404 12118 raft_consensus.cc:2258] T adfa098aa30244478834002fa731070e P 578ae1f534534b3aa0585dcbd0781ede [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 04:57:20.800503 12118 ts_tablet_manager.cc:1905] T adfa098aa30244478834002fa731070e P 578ae1f534534b3aa0585dcbd0781ede: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 04:57:20.800534 12119 ts_tablet_manager.cc:1905] T adfa098aa30244478834002fa731070e P bb34b1fe2b6a439fb22a40a0e22d9607: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 04:57:20.803699 12117 tablet_replica.cc:330] T adfa098aa30244478834002fa731070e P 3c92b56d4ab3443b9c85bdabf4a72977: stopping tablet replica
I20260806 04:57:20.803960 12117 raft_consensus.cc:2229] T adfa098aa30244478834002fa731070e P 3c92b56d4ab3443b9c85bdabf4a72977 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 04:57:20.804340 12117 raft_consensus.cc:2258] T adfa098aa30244478834002fa731070e P 3c92b56d4ab3443b9c85bdabf4a72977 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 04:57:20.805742 12117 ts_tablet_manager.cc:1905] T adfa098aa30244478834002fa731070e P 3c92b56d4ab3443b9c85bdabf4a72977: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 04:57:20.808164 12118 ts_tablet_manager.cc:1918] T adfa098aa30244478834002fa731070e P 578ae1f534534b3aa0585dcbd0781ede: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 04:57:20.808475 12118 log.cc:1196] T adfa098aa30244478834002fa731070e P 578ae1f534534b3aa0585dcbd0781ede: Deleting WAL directory at /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785992227050689-9601-0/minicluster-data/ts-2/wal/wals/adfa098aa30244478834002fa731070e
I20260806 04:57:20.809072 12118 ts_tablet_manager.cc:1939] T adfa098aa30244478834002fa731070e P 578ae1f534534b3aa0585dcbd0781ede: Deleting consensus metadata
I20260806 04:57:20.810376 11239 catalog_manager.cc:4841] TS 578ae1f534534b3aa0585dcbd0781ede (127.9.96.67:32807): tablet adfa098aa30244478834002fa731070e (table test-table [id=10a4a48105614cb395d55c24b097deaf]) successfully deleted
I20260806 04:57:20.811512 12119 ts_tablet_manager.cc:1918] T adfa098aa30244478834002fa731070e P bb34b1fe2b6a439fb22a40a0e22d9607: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 04:57:20.811738 12119 log.cc:1196] T adfa098aa30244478834002fa731070e P bb34b1fe2b6a439fb22a40a0e22d9607: Deleting WAL directory at /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785992227050689-9601-0/minicluster-data/ts-0/wal/wals/adfa098aa30244478834002fa731070e
I20260806 04:57:20.812424 12119 ts_tablet_manager.cc:1939] T adfa098aa30244478834002fa731070e P bb34b1fe2b6a439fb22a40a0e22d9607: Deleting consensus metadata
I20260806 04:57:20.813777 11239 catalog_manager.cc:4841] TS bb34b1fe2b6a439fb22a40a0e22d9607 (127.9.96.65:39511): tablet adfa098aa30244478834002fa731070e (table test-table [id=10a4a48105614cb395d55c24b097deaf]) successfully deleted
I20260806 04:57:20.815744 12117 ts_tablet_manager.cc:1918] T adfa098aa30244478834002fa731070e P 3c92b56d4ab3443b9c85bdabf4a72977: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 04:57:20.815918 12117 log.cc:1196] T adfa098aa30244478834002fa731070e P 3c92b56d4ab3443b9c85bdabf4a72977: Deleting WAL directory at /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785992227050689-9601-0/minicluster-data/ts-1/wal/wals/adfa098aa30244478834002fa731070e
I20260806 04:57:20.816497 12117 ts_tablet_manager.cc:1939] T adfa098aa30244478834002fa731070e P 3c92b56d4ab3443b9c85bdabf4a72977: Deleting consensus metadata
I20260806 04:57:20.817836 11243 catalog_manager.cc:4841] TS 3c92b56d4ab3443b9c85bdabf4a72977 (127.9.96.66:42211): tablet adfa098aa30244478834002fa731070e (table test-table [id=10a4a48105614cb395d55c24b097deaf]) successfully deleted
Aug 06 04:57:20 dist-test-slave-x4qh krb5kdc[11195](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992239, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.9.96.65@KRBTEST.COM
I20260806 04:57:20.836966 12092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:57:20.808630 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34358 (local address 127.9.96.65:39511)
0806 04:57:20.808852 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0806 04:57:20.808859 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 04:57:20.814810 (+ 5951us) server_negotiation.cc:407] Connection header received
0806 04:57:20.815042 (+ 232us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:57:20.815050 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:57:20.815160 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:57:20.815285 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:57:20.817720 (+ 2435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:57:20.818770 (+ 1050us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:57:20.820892 (+ 2122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:57:20.821410 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:57:20.821532 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:57:20.830735 (+ 9203us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:57:20.830748 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:57:20.830764 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:57:20.830801 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:57:20.833956 (+ 3155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:57:20.834711 (+ 755us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:57:20.834719 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:57:20.834721 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:57:20.834878 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:57:20.835282 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:57:20.835292 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:57:20.835294 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:57:20.835707 (+ 413us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:57:20.836043 (+ 336us) server_negotiation.cc:1035] Waiting for connection context
0806 04:57:20.836352 (+ 309us) server_negotiation.cc:299] Negotiation successful
0806 04:57:20.836586 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
W20260806 04:57:20.837992 11565 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34358
I20260806 04:57:20.856338 12069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:57:20.841598 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43184 (local address 127.9.96.126:34519)
0806 04:57:20.841800 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0806 04:57:20.841805 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 04:57:20.842026 (+ 221us) server_negotiation.cc:407] Connection header received
0806 04:57:20.842144 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:57:20.842150 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:57:20.842241 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:57:20.842349 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:57:20.843625 (+ 1276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:57:20.845138 (+ 1513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:57:20.846512 (+ 1374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:57:20.847201 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:57:20.847330 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:57:20.849742 (+ 2412us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:57:20.849749 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:57:20.849753 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:57:20.849791 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:57:20.853283 (+ 3492us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:57:20.854043 (+ 760us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:57:20.854049 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:57:20.854052 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:57:20.854235 (+ 183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:57:20.854670 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:57:20.854675 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:57:20.854679 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:57:20.855083 (+ 404us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:57:20.855354 (+ 271us) server_negotiation.cc:1035] Waiting for connection context
0806 04:57:20.855749 (+ 395us) server_negotiation.cc:299] Negotiation successful
0806 04:57:20.855996 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
W20260806 04:57:20.856962 11264 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43184
I20260806 04:57:20.859596 9601 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu pid 11373
I20260806 04:57:20.873620 12092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:57:20.860440 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34372 (local address 127.9.96.65:39511)
0806 04:57:20.860718 (+ 278us) server_negotiation.cc:206] Beginning negotiation
0806 04:57:20.860732 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0806 04:57:20.861225 (+ 493us) server_negotiation.cc:407] Connection header received
0806 04:57:20.861435 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:57:20.861441 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:57:20.861515 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:57:20.861625 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:57:20.862723 (+ 1098us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:57:20.863531 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:57:20.864743 (+ 1212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:57:20.865256 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:57:20.865455 (+ 199us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:57:20.867738 (+ 2283us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:57:20.867745 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:57:20.867749 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:57:20.867790 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:57:20.870593 (+ 2803us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:57:20.871225 (+ 632us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:57:20.871232 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:57:20.871236 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:57:20.871420 (+ 184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:57:20.871805 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:57:20.871811 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:57:20.871814 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:57:20.872269 (+ 455us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:57:20.872621 (+ 352us) server_negotiation.cc:1035] Waiting for connection context
0806 04:57:20.872908 (+ 287us) server_negotiation.cc:299] Negotiation successful
0806 04:57:20.873185 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
W20260806 04:57:20.874217 11565 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34372
W20260806 04:57:20.874791 9601 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu pid 11373: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260806 04:57:20.875010 9601 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu with pid 11373
I20260806 04:57:20.905521 9601 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu pid 11582
Aug 06 04:57:20 dist-test-slave-x4qh krb5kdc[11195](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992239, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.9.96.66@KRBTEST.COM
I20260806 04:57:20.922508 12077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:57:20.906374 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53044 (local address 127.9.96.66:42211)
0806 04:57:20.906579 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0806 04:57:20.906584 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 04:57:20.906600 (+ 16us) server_negotiation.cc:407] Connection header received
0806 04:57:20.906658 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:57:20.906661 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:57:20.906733 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:57:20.906844 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:57:20.908502 (+ 1658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:57:20.909221 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:57:20.911044 (+ 1823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:57:20.911555 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:57:20.911693 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:57:20.916477 (+ 4784us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:57:20.916491 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:57:20.916507 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:57:20.916540 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:57:20.919584 (+ 3044us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:57:20.920330 (+ 746us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:57:20.920338 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:57:20.920341 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:57:20.920492 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:57:20.920911 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:57:20.920915 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:57:20.920917 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:57:20.921313 (+ 396us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:57:20.921621 (+ 308us) server_negotiation.cc:1035] Waiting for connection context
0806 04:57:20.921903 (+ 282us) server_negotiation.cc:299] Negotiation successful
0806 04:57:20.922175 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
W20260806 04:57:20.923341 11781 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53044
W20260806 04:57:20.924012 9601 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu pid 11582: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260806 04:57:20.924206 9601 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu with pid 11582
I20260806 04:57:20.948891 9601 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu pid 11792
Aug 06 04:57:20 dist-test-slave-x4qh krb5kdc[11195](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992239, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.9.96.67@KRBTEST.COM
I20260806 04:57:20.964128 12004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:57:20.949654 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45368 (local address 127.9.96.67:32807)
0806 04:57:20.949877 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0806 04:57:20.949882 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 04:57:20.949903 (+ 21us) server_negotiation.cc:407] Connection header received
0806 04:57:20.949987 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:57:20.949991 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:57:20.950068 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:57:20.950162 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:57:20.951455 (+ 1293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:57:20.952306 (+ 851us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:57:20.953472 (+ 1166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:57:20.953998 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:57:20.954113 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:57:20.958419 (+ 4306us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:57:20.958435 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:57:20.958451 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:57:20.958487 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:57:20.961525 (+ 3038us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:57:20.962230 (+ 705us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:57:20.962239 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:57:20.962241 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:57:20.962397 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:57:20.962731 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:57:20.962735 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:57:20.962737 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:57:20.963104 (+ 367us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:57:20.963486 (+ 382us) server_negotiation.cc:1035] Waiting for connection context
0806 04:57:20.963630 (+ 144us) server_negotiation.cc:299] Negotiation successful
0806 04:57:20.963837 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":94}
W20260806 04:57:20.964886 11976 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45368
W20260806 04:57:20.965586 9601 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu pid 11792: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260806 04:57:20.965763 9601 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu with pid 11792
I20260806 04:57:20.996393 9601 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu pid 11215
I20260806 04:57:20.997634 11328 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 04:57:21.113026 9601 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu with pid 11215
2026-08-06T04:57:21Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (5091 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785992227050689-9601-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 04:57:21 dist-test-slave-x4qh krb5kdc[12131](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 04:57:21 dist-test-slave-x4qh krb5kdc[12131](info): set up 2 sockets
Aug 06 04:57:21 dist-test-slave-x4qh krb5kdc[12131](info): commencing operation
krb5kdc: starting...
W20260806 04:57:22.395805 9601 mini_kdc.cc:121] Time spent starting KDC: real 1.228s user 0.000s sys 0.019s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785992227050689-9601-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785992227050689-9601-0/krb5kdc/test-user.keytab.
Aug 06 04:57:22 dist-test-slave-x4qh krb5kdc[12131](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992242, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T04:57:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T04:57:22Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.96.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.96.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.96.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.126.keytab.
Entry for principal kudu/127.9.96.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.126.keytab.
I20260806 04:57:22.588564 9601 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
/tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785992227050689-9601-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785992227050689-9601-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785992227050689-9601-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785992227050689-9601-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.96.126:39063
--webserver_interface=127.9.96.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.126.keytab
--principal=kudu/127.9.96.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.96.84:38505
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.96.126:39063
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785992227050689-9601-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785992227050689-9601-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785992227050689-9601-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 04:57:22.906999 12159 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 04:57:22.907277 12159 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 04:57:22.907416 12159 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 04:57:22.914395 12159 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 04:57:22.914487 12159 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260806 04:57:22.914527 12159 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 04:57:22.914597 12159 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 04:57:22.914644 12159 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 04:57:22.914697 12159 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 04:57:22.924319 12159 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.96.84:38505
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785992227050689-9601-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785992227050689-9601-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.96.126:39063
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.126.keytab
--principal=kudu/127.9.96.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.96.126:39063
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785992227050689-9601-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.96.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785992227050689-9601-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 03:56:12 UTC on ccycloud.quasar-guijbm.root.comops.site
build id 4569618
ASAN enabled
I20260806 04:57:22.925516 12159 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 04:57:22.926748 12159 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 04:57:22.938331 12167 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:57:22.938376 12170 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:57:22.938421 12168 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:57:22.939627 12159 server_base.cc:1034] running on GCE node
I20260806 04:57:22.940382 12159 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 04:57:22.941716 12159 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 04:57:22.942970 12159 hybrid_clock.cc:648] HybridClock initialized: now 1785992242942920 us; error 44 us; skew 500 ppm
I20260806 04:57:22.943328 12159 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 04:57:22 dist-test-slave-x4qh krb5kdc[12131](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992242, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.96.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 04:57:22.946542 12159 init.cc:377] Logged in from keytab as kudu/127.9.96.126@KRBTEST.COM (short username kudu)
I20260806 04:57:22.948024 12159 webserver.cc:460] Webserver started at http://127.9.96.126:34017/ using document root <none> and password file <none>
I20260806 04:57:22.948657 12159 fs_manager.cc:359] Metadata directory not provided
I20260806 04:57:22.948760 12159 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 04:57:22.949018 12159 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 04:57:22.951937 12159 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785992227050689-9601-0/minicluster-data/master-0/data/instance:
uuid: "ebfef4bb062640d9bf626f0e62f6b98c"
format_stamp: "Formatted at 2026-08-06 04:57:22 on dist-test-slave-x4qh"
server_key: "b26ef9751e50cdb1dc9ed49214674072"
server_key_iv: "47e9dd4f38b32ac60b57d025cd05e886"
server_key_version: "encryptionkey@0"
I20260806 04:57:22.952644 12159 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785992227050689-9601-0/minicluster-data/master-0/wal/instance:
uuid: "ebfef4bb062640d9bf626f0e62f6b98c"
format_stamp: "Formatted at 2026-08-06 04:57:22 on dist-test-slave-x4qh"
server_key: "b26ef9751e50cdb1dc9ed49214674072"
server_key_iv: "47e9dd4f38b32ac60b57d025cd05e886"
server_key_version: "encryptionkey@0"
I20260806 04:57:22.957697 12159 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260806 04:57:22.960915 12183 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 04:57:22.962126 12159 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260806 04:57:22.962313 12159 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785992227050689-9601-0/minicluster-data/master-0/data,/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785992227050689-9601-0/minicluster-data/master-0/wal
uuid: "ebfef4bb062640d9bf626f0e62f6b98c"
format_stamp: "Formatted at 2026-08-06 04:57:22 on dist-test-slave-x4qh"
server_key: "b26ef9751e50cdb1dc9ed49214674072"
server_key_iv: "47e9dd4f38b32ac60b57d025cd05e886"
server_key_version: "encryptionkey@0"
I20260806 04:57:22.962514 12159 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785992227050689-9601-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785992227050689-9601-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785992227050689-9601-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 04:57:22.974927 12159 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 04:57:22.979296 12159 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 04:57:22.979678 12159 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 04:57:23.001015 12159 rpc_server.cc:307] RPC server started. Bound to: 127.9.96.126:39063
I20260806 04:57:23.001022 12268 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.96.126:39063 every 8 connection(s)
I20260806 04:57:23.002383 12159 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785992227050689-9601-0/minicluster-data/master-0/data/info.pb
I20260806 04:57:23.004701 9601 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu as pid 12159
I20260806 04:57:23.004925 9601 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785992227050689-9601-0/minicluster-data/master-0/wal/instance
I20260806 04:57:23.005610 9601 external_mini_cluster.cc:1428] Setting key 9844d35f347ae79bf6b4feb83e4d6a58
I20260806 04:57:23.006378 12269 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 04:57:23.020797 12269 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ebfef4bb062640d9bf626f0e62f6b98c: Bootstrap starting.
Aug 06 04:57:23 dist-test-slave-x4qh krb5kdc[12131](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992242, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.96.126@KRBTEST.COM
I20260806 04:57:23.024914 12269 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ebfef4bb062640d9bf626f0e62f6b98c: Neither blocks nor log segments found. Creating new log.
I20260806 04:57:23.026247 12269 log.cc:824] T 00000000000000000000000000000000 P ebfef4bb062640d9bf626f0e62f6b98c: Log is configured to *not* fsync() on all Append() calls
I20260806 04:57:23.029860 12269 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ebfef4bb062640d9bf626f0e62f6b98c: No bootstrap required, opened a new log
I20260806 04:57:23.031680 12273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:57:23.008067 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47860 (local address 127.9.96.126:39063)
0806 04:57:23.008853 (+ 786us) server_negotiation.cc:206] Beginning negotiation
0806 04:57:23.008869 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 04:57:23.008919 (+ 50us) server_negotiation.cc:407] Connection header received
0806 04:57:23.011251 (+ 2332us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:57:23.011278 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:57:23.011798 (+ 520us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:57:23.012258 (+ 460us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:57:23.013775 (+ 1517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:57:23.015741 (+ 1966us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:57:23.016981 (+ 1240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:57:23.018094 (+ 1113us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:57:23.018285 (+ 191us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:57:23.022745 (+ 4460us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:57:23.022759 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:57:23.022783 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:57:23.022834 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:57:23.026894 (+ 4060us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:57:23.027958 (+ 1064us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:57:23.027966 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:57:23.027974 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:57:23.028173 (+ 199us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:57:23.028547 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:57:23.028551 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:57:23.028555 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:57:23.029256 (+ 701us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:57:23.029550 (+ 294us) server_negotiation.cc:1035] Waiting for connection context
0806 04:57:23.029946 (+ 396us) server_negotiation.cc:299] Negotiation successful
0806 04:57:23.030380 (+ 434us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":559,"thread_start_us":351,"threads_started":1}
I20260806 04:57:23.038511 12269 raft_consensus.cc:348] T 00000000000000000000000000000000 P ebfef4bb062640d9bf626f0e62f6b98c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ebfef4bb062640d9bf626f0e62f6b98c" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 39063 } }
I20260806 04:57:23.038944 12269 raft_consensus.cc:374] T 00000000000000000000000000000000 P ebfef4bb062640d9bf626f0e62f6b98c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 04:57:23.039048 12269 raft_consensus.cc:729] T 00000000000000000000000000000000 P ebfef4bb062640d9bf626f0e62f6b98c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ebfef4bb062640d9bf626f0e62f6b98c, State: Initialized, Role: FOLLOWER
I20260806 04:57:23.039678 12269 consensus_queue.cc:260] T 00000000000000000000000000000000 P ebfef4bb062640d9bf626f0e62f6b98c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ebfef4bb062640d9bf626f0e62f6b98c" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 39063 } }
I20260806 04:57:23.039917 12269 raft_consensus.cc:388] T 00000000000000000000000000000000 P ebfef4bb062640d9bf626f0e62f6b98c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 04:57:23.040053 12269 raft_consensus.cc:482] T 00000000000000000000000000000000 P ebfef4bb062640d9bf626f0e62f6b98c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 04:57:23.040243 12269 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ebfef4bb062640d9bf626f0e62f6b98c [term 0 FOLLOWER]: Advancing to term 1
I20260806 04:57:23.042327 12269 raft_consensus.cc:504] T 00000000000000000000000000000000 P ebfef4bb062640d9bf626f0e62f6b98c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ebfef4bb062640d9bf626f0e62f6b98c" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 39063 } }
I20260806 04:57:23.042807 12269 leader_election.cc:302] T 00000000000000000000000000000000 P ebfef4bb062640d9bf626f0e62f6b98c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ebfef4bb062640d9bf626f0e62f6b98c; no voters:
I20260806 04:57:23.043449 12269 leader_election.cc:288] T 00000000000000000000000000000000 P ebfef4bb062640d9bf626f0e62f6b98c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 04:57:23.043571 12276 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ebfef4bb062640d9bf626f0e62f6b98c [term 1 FOLLOWER]: Leader election won for term 1
I20260806 04:57:23.044327 12276 raft_consensus.cc:686] T 00000000000000000000000000000000 P ebfef4bb062640d9bf626f0e62f6b98c [term 1 LEADER]: Becoming Leader. State: Replica: ebfef4bb062640d9bf626f0e62f6b98c, State: Running, Role: LEADER
I20260806 04:57:23.044917 12276 consensus_queue.cc:237] T 00000000000000000000000000000000 P ebfef4bb062640d9bf626f0e62f6b98c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ebfef4bb062640d9bf626f0e62f6b98c" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 39063 } }
I20260806 04:57:23.045696 12269 sys_catalog.cc:564] T 00000000000000000000000000000000 P ebfef4bb062640d9bf626f0e62f6b98c [sys.catalog]: configured and running, proceeding with master startup.
I20260806 04:57:23.047839 12277 sys_catalog.cc:455] T 00000000000000000000000000000000 P ebfef4bb062640d9bf626f0e62f6b98c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ebfef4bb062640d9bf626f0e62f6b98c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ebfef4bb062640d9bf626f0e62f6b98c" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 39063 } } }
I20260806 04:57:23.048274 12277 sys_catalog.cc:458] T 00000000000000000000000000000000 P ebfef4bb062640d9bf626f0e62f6b98c [sys.catalog]: This master's current role is: LEADER
I20260806 04:57:23.048975 12278 sys_catalog.cc:455] T 00000000000000000000000000000000 P ebfef4bb062640d9bf626f0e62f6b98c [sys.catalog]: SysCatalogTable state changed. Reason: New leader ebfef4bb062640d9bf626f0e62f6b98c. Latest consensus state: current_term: 1 leader_uuid: "ebfef4bb062640d9bf626f0e62f6b98c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ebfef4bb062640d9bf626f0e62f6b98c" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 39063 } } }
I20260806 04:57:23.049252 12278 sys_catalog.cc:458] T 00000000000000000000000000000000 P ebfef4bb062640d9bf626f0e62f6b98c [sys.catalog]: This master's current role is: LEADER
I20260806 04:57:23.050034 12285 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 04:57:23.055056 12285 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 04:57:23.063100 12285 catalog_manager.cc:1347] Generated new cluster ID: 0af691145a7843bd851a917fa75fe155
I20260806 04:57:23.063246 12285 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 04:57:23.076133 12285 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 04:57:23.077906 12285 catalog_manager.cc:1504] Loading token signing keys...
I20260806 04:57:23.095315 12285 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ebfef4bb062640d9bf626f0e62f6b98c: Generated new TSK 0
I20260806 04:57:23.096125 12285 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.9.96.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.96.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.96.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.65.keytab.
Entry for principal kudu/127.9.96.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.65.keytab.
I20260806 04:57:23.143190 9601 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
/tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785992227050689-9601-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785992227050689-9601-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785992227050689-9601-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785992227050689-9601-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.96.65:0
--local_ip_for_outbound_sockets=127.9.96.65
--webserver_interface=127.9.96.65
--webserver_port=0
--tserver_master_addrs=127.9.96.126:39063
--keytab_file=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.65.keytab
--principal=kudu/127.9.96.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.96.84:38505
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785992227050689-9601-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785992227050689-9601-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785992227050689-9601-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 04:57:23.468286 12306 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 04:57:23.468523 12306 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 04:57:23.468691 12306 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 04:57:23.475310 12306 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 04:57:23.475416 12306 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 04:57:23.475564 12306 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260806 04:57:23.475643 12306 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 04:57:23.475787 12306 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.96.65
I20260806 04:57:23.484707 12306 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.96.84:38505
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785992227050689-9601-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785992227050689-9601-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.65.keytab
--principal=kudu/127.9.96.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.96.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785992227050689-9601-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.96.65
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.9.96.126:39063
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.96.65
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785992227050689-9601-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 03:56:12 UTC on ccycloud.quasar-guijbm.root.comops.site
build id 4569618
ASAN enabled
I20260806 04:57:23.485821 12306 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 04:57:23.487099 12306 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 04:57:23.500139 12313 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:57:23.500239 12316 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:57:23.500329 12314 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:57:23.501031 12306 server_base.cc:1034] running on GCE node
I20260806 04:57:23.501642 12306 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 04:57:23.502476 12306 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 04:57:23.503711 12306 hybrid_clock.cc:648] HybridClock initialized: now 1785992243503658 us; error 45 us; skew 500 ppm
I20260806 04:57:23.504010 12306 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 04:57:23 dist-test-slave-x4qh krb5kdc[12131](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992243, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.96.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 04:57:23.508054 12306 init.cc:377] Logged in from keytab as kudu/127.9.96.65@KRBTEST.COM (short username kudu)
I20260806 04:57:23.510069 12306 webserver.cc:460] Webserver started at http://127.9.96.65:42241/ using document root <none> and password file <none>
I20260806 04:57:23.510926 12306 fs_manager.cc:359] Metadata directory not provided
I20260806 04:57:23.511094 12306 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 04:57:23.511503 12306 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 04:57:23.515200 12306 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785992227050689-9601-0/minicluster-data/ts-0/data/instance:
uuid: "4b26fa9b4a984025a159d0f38be81428"
format_stamp: "Formatted at 2026-08-06 04:57:23 on dist-test-slave-x4qh"
server_key: "34092791b84a1e2fa84718abb85a09e7"
server_key_iv: "8062a5f25170ccf4deab9c95ebfa65d4"
server_key_version: "encryptionkey@0"
I20260806 04:57:23.516093 12306 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785992227050689-9601-0/minicluster-data/ts-0/wal/instance:
uuid: "4b26fa9b4a984025a159d0f38be81428"
format_stamp: "Formatted at 2026-08-06 04:57:23 on dist-test-slave-x4qh"
server_key: "34092791b84a1e2fa84718abb85a09e7"
server_key_iv: "8062a5f25170ccf4deab9c95ebfa65d4"
server_key_version: "encryptionkey@0"
I20260806 04:57:23.521149 12306 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260806 04:57:23.524398 12332 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 04:57:23.525633 12306 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260806 04:57:23.525811 12306 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785992227050689-9601-0/minicluster-data/ts-0/data,/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785992227050689-9601-0/minicluster-data/ts-0/wal
uuid: "4b26fa9b4a984025a159d0f38be81428"
format_stamp: "Formatted at 2026-08-06 04:57:23 on dist-test-slave-x4qh"
server_key: "34092791b84a1e2fa84718abb85a09e7"
server_key_iv: "8062a5f25170ccf4deab9c95ebfa65d4"
server_key_version: "encryptionkey@0"
I20260806 04:57:23.526012 12306 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785992227050689-9601-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785992227050689-9601-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785992227050689-9601-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 04:57:23.544881 12306 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 04:57:23.548895 12306 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 04:57:23.549180 12306 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 04:57:23.552059 12306 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 04:57:23.552206 12306 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 04:57:23.552333 12306 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 04:57:23.552502 12306 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.001s sys 0.000s
Aug 06 04:57:23 dist-test-slave-x4qh krb5kdc[12131](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992243, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.96.65@KRBTEST.COM for kudu/127.9.96.126@KRBTEST.COM
I20260806 04:57:23.580475 12348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:57:23.553531 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:39063 (local address 127.9.96.65:52577)
0806 04:57:23.554624 (+ 1093us) negotiation.cc:107] Waiting for socket to connect
0806 04:57:23.554688 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0806 04:57:23.557698 (+ 3010us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:57:23.558674 (+ 976us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:57:23.558689 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:57:23.559623 (+ 934us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:57:23.560724 (+ 1101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:57:23.560757 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:57:23.561932 (+ 1175us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:57:23.561941 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:57:23.564997 (+ 3056us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:57:23.565023 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:57:23.565957 (+ 934us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:57:23.565971 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:57:23.566351 (+ 380us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:57:23.567269 (+ 918us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:57:23.567311 (+ 42us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:57:23.571590 (+ 4279us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 04:57:23.575346 (+ 3756us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:57:23.575358 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:57:23.575391 (+ 33us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:57:23.576066 (+ 675us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:57:23.576573 (+ 507us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:57:23.576578 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:57:23.576582 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:57:23.576912 (+ 330us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:57:23.577772 (+ 860us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 04:57:23.577778 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 04:57:23.578266 (+ 488us) client_negotiation.cc:769] Sending connection context
0806 04:57:23.578692 (+ 426us) client_negotiation.cc:240] Negotiation successful
0806 04:57:23.579238 (+ 546us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":778,"thread_start_us":482,"threads_started":1}
I20260806 04:57:23.580528 12347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:57:23.552723 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.96.65:52577 (local address 127.9.96.126:39063)
0806 04:57:23.555738 (+ 3015us) server_negotiation.cc:206] Beginning negotiation
0806 04:57:23.555743 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 04:57:23.555761 (+ 18us) server_negotiation.cc:407] Connection header received
0806 04:57:23.558100 (+ 2339us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:57:23.558108 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:57:23.558224 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:57:23.558354 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:57:23.561025 (+ 2671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:57:23.561738 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:57:23.565171 (+ 3433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:57:23.565620 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:57:23.565771 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:57:23.571899 (+ 6128us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:57:23.571907 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:57:23.571915 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:57:23.571963 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:57:23.575174 (+ 3211us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:57:23.576294 (+ 1120us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:57:23.576302 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:57:23.576308 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:57:23.576469 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:57:23.577103 (+ 634us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:57:23.577110 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:57:23.577114 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:57:23.577515 (+ 401us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:57:23.577692 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0806 04:57:23.579636 (+ 1944us) server_negotiation.cc:299] Negotiation successful
0806 04:57:23.579957 (+ 321us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2894,"thread_start_us":2809,"threads_started":1}
I20260806 04:57:23.600720 12347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:57:23.586862 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.96.65:42283 (local address 127.9.96.126:39063)
0806 04:57:23.587056 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0806 04:57:23.587062 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 04:57:23.587750 (+ 688us) server_negotiation.cc:407] Connection header received
0806 04:57:23.587952 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:57:23.587958 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:57:23.588033 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:57:23.588134 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:57:23.589061 (+ 927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:57:23.589778 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:57:23.591038 (+ 1260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:57:23.591581 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:57:23.591713 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:57:23.594043 (+ 2330us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:57:23.594048 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:57:23.594051 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:57:23.594079 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:57:23.597535 (+ 3456us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:57:23.598489 (+ 954us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:57:23.598494 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:57:23.598497 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:57:23.598641 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:57:23.599128 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:57:23.599133 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:57:23.599136 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:57:23.599510 (+ 374us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:57:23.599690 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0806 04:57:23.600049 (+ 359us) server_negotiation.cc:299] Negotiation successful
0806 04:57:23.600307 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260806 04:57:23.600768 12442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:57:23.586918 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:39063 (local address 127.9.96.65:42283)
0806 04:57:23.587603 (+ 685us) negotiation.cc:107] Waiting for socket to connect
0806 04:57:23.587620 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 04:57:23.587822 (+ 202us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:57:23.588161 (+ 339us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:57:23.588165 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:57:23.588625 (+ 460us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:57:23.588921 (+ 296us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:57:23.588930 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:57:23.589920 (+ 990us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:57:23.589926 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:57:23.590903 (+ 977us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:57:23.590914 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:57:23.591786 (+ 872us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:57:23.591794 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:57:23.591979 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:57:23.592682 (+ 703us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:57:23.592700 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:57:23.593775 (+ 1075us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 04:57:23.597728 (+ 3953us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:57:23.597736 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:57:23.597744 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:57:23.598301 (+ 557us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:57:23.598769 (+ 468us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:57:23.598774 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:57:23.598777 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:57:23.598955 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:57:23.599639 (+ 684us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 04:57:23.599642 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 04:57:23.599800 (+ 158us) client_negotiation.cc:769] Sending connection context
0806 04:57:23.599918 (+ 118us) client_negotiation.cc:240] Negotiation successful
0806 04:57:23.600132 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":565,"thread_start_us":365,"threads_started":1}
I20260806 04:57:23.611819 12306 rpc_server.cc:307] RPC server started. Bound to: 127.9.96.65:41697
I20260806 04:57:23.611896 12506 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.96.65:41697 every 8 connection(s)
I20260806 04:57:23.612998 12306 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785992227050689-9601-0/minicluster-data/ts-0/data/info.pb
I20260806 04:57:23.614890 12507 heartbeater.cc:347] Connected to a master server at 127.9.96.126:39063
I20260806 04:57:23.615131 12507 heartbeater.cc:464] Registering TS with master...
I20260806 04:57:23.615831 12507 heartbeater.cc:511] Master 127.9.96.126:39063 requested a full tablet report, sending...
I20260806 04:57:23.617663 12210 ts_manager.cc:194] Registered new tserver with Master: 4b26fa9b4a984025a159d0f38be81428 (127.9.96.65:41697)
I20260806 04:57:23.619077 12210 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.96.65@KRBTEST.COM'} at 127.9.96.65:52577
I20260806 04:57:23.622150 9601 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu as pid 12306
I20260806 04:57:23.622372 9601 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785992227050689-9601-0/minicluster-data/ts-0/wal/instance
I20260806 04:57:23.623031 9601 external_mini_cluster.cc:1428] Setting key 1e230dbb92603405826d3281927023cd
WARNING: no policy specified for kudu/127.9.96.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.96.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.96.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.66.keytab.
Entry for principal kudu/127.9.96.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.66.keytab.
I20260806 04:57:23.670742 9601 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
/tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785992227050689-9601-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785992227050689-9601-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785992227050689-9601-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785992227050689-9601-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.96.66:0
--local_ip_for_outbound_sockets=127.9.96.66
--webserver_interface=127.9.96.66
--webserver_port=0
--tserver_master_addrs=127.9.96.126:39063
--keytab_file=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.66.keytab
--principal=kudu/127.9.96.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.96.84:38505
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785992227050689-9601-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785992227050689-9601-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785992227050689-9601-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 04:57:23.990715 12512 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 04:57:23.990984 12512 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 04:57:23.991092 12512 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 04:57:23.998170 12512 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 04:57:23.998270 12512 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 04:57:23.998353 12512 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260806 04:57:23.998407 12512 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 04:57:23.998467 12512 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.96.66
I20260806 04:57:24.008123 12512 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.96.84:38505
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785992227050689-9601-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785992227050689-9601-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.66.keytab
--principal=kudu/127.9.96.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.96.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785992227050689-9601-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.96.66
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.9.96.126:39063
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.96.66
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785992227050689-9601-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 03:56:12 UTC on ccycloud.quasar-guijbm.root.comops.site
build id 4569618
ASAN enabled
I20260806 04:57:24.009379 12512 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 04:57:24.010681 12512 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 04:57:24.024160 12526 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:57:24.024233 12520 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:57:24.024698 12518 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:57:24.025296 12512 server_base.cc:1034] running on GCE node
I20260806 04:57:24.025914 12512 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 04:57:24.026834 12512 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 04:57:24.028057 12512 hybrid_clock.cc:648] HybridClock initialized: now 1785992244028018 us; error 36 us; skew 500 ppm
I20260806 04:57:24.028379 12512 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 04:57:24 dist-test-slave-x4qh krb5kdc[12131](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992244, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.96.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 04:57:24.031608 12512 init.cc:377] Logged in from keytab as kudu/127.9.96.66@KRBTEST.COM (short username kudu)
I20260806 04:57:24.033125 12512 webserver.cc:460] Webserver started at http://127.9.96.66:44805/ using document root <none> and password file <none>
I20260806 04:57:24.033812 12512 fs_manager.cc:359] Metadata directory not provided
I20260806 04:57:24.033962 12512 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 04:57:24.034300 12512 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 04:57:24.037211 12512 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785992227050689-9601-0/minicluster-data/ts-1/data/instance:
uuid: "61f45f84da5f4e89bd2a27ed0cdc051b"
format_stamp: "Formatted at 2026-08-06 04:57:24 on dist-test-slave-x4qh"
server_key: "f500bc3ad9799f4526a91d75f3869f92"
server_key_iv: "76bb83b6c2428e77fa9228f336f8725d"
server_key_version: "encryptionkey@0"
I20260806 04:57:24.037963 12512 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785992227050689-9601-0/minicluster-data/ts-1/wal/instance:
uuid: "61f45f84da5f4e89bd2a27ed0cdc051b"
format_stamp: "Formatted at 2026-08-06 04:57:24 on dist-test-slave-x4qh"
server_key: "f500bc3ad9799f4526a91d75f3869f92"
server_key_iv: "76bb83b6c2428e77fa9228f336f8725d"
server_key_version: "encryptionkey@0"
I20260806 04:57:24.042882 12512 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260806 04:57:24.046406 12537 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 04:57:24.047799 12512 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 04:57:24.047982 12512 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785992227050689-9601-0/minicluster-data/ts-1/data,/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785992227050689-9601-0/minicluster-data/ts-1/wal
uuid: "61f45f84da5f4e89bd2a27ed0cdc051b"
format_stamp: "Formatted at 2026-08-06 04:57:24 on dist-test-slave-x4qh"
server_key: "f500bc3ad9799f4526a91d75f3869f92"
server_key_iv: "76bb83b6c2428e77fa9228f336f8725d"
server_key_version: "encryptionkey@0"
I20260806 04:57:24.048189 12512 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785992227050689-9601-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785992227050689-9601-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785992227050689-9601-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 04:57:24.083658 12512 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 04:57:24.087802 12512 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 04:57:24.088104 12512 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 04:57:24.090684 12512 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 04:57:24.090842 12512 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 04:57:24.091001 12512 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 04:57:24.091092 12512 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 04:57:24 dist-test-slave-x4qh krb5kdc[12131](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992244, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.96.66@KRBTEST.COM for kudu/127.9.96.126@KRBTEST.COM
I20260806 04:57:24.114809 12347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:57:24.091632 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.96.66:57477 (local address 127.9.96.126:39063)
0806 04:57:24.091864 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0806 04:57:24.091871 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 04:57:24.093633 (+ 1762us) server_negotiation.cc:407] Connection header received
0806 04:57:24.096102 (+ 2469us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:57:24.096108 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:57:24.096208 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:57:24.096323 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:57:24.097919 (+ 1596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:57:24.098976 (+ 1057us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:57:24.101542 (+ 2566us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:57:24.102194 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:57:24.102335 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:57:24.107466 (+ 5131us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:57:24.107476 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:57:24.107482 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:57:24.107523 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:57:24.111027 (+ 3504us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:57:24.111989 (+ 962us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:57:24.111996 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:57:24.112003 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:57:24.112167 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:57:24.112849 (+ 682us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:57:24.112854 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:57:24.112857 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:57:24.113218 (+ 361us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:57:24.113385 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0806 04:57:24.114374 (+ 989us) server_negotiation.cc:299] Negotiation successful
0806 04:57:24.114532 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
I20260806 04:57:24.116860 12553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:57:24.092484 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:39063 (local address 127.9.96.66:57477)
0806 04:57:24.093439 (+ 955us) negotiation.cc:107] Waiting for socket to connect
0806 04:57:24.093507 (+ 68us) client_negotiation.cc:174] Beginning negotiation
0806 04:57:24.095822 (+ 2315us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:57:24.096581 (+ 759us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:57:24.096595 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:57:24.097190 (+ 595us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:57:24.097729 (+ 539us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:57:24.097746 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:57:24.099160 (+ 1414us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:57:24.099167 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:57:24.101366 (+ 2199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:57:24.101386 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:57:24.102466 (+ 1080us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:57:24.102478 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:57:24.102831 (+ 353us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:57:24.103665 (+ 834us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:57:24.103703 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:57:24.107179 (+ 3476us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 04:57:24.111230 (+ 4051us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:57:24.111241 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:57:24.111261 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:57:24.111789 (+ 528us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:57:24.112316 (+ 527us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:57:24.112320 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:57:24.112324 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:57:24.112673 (+ 349us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:57:24.113395 (+ 722us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 04:57:24.113406 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 04:57:24.113925 (+ 519us) client_negotiation.cc:769] Sending connection context
0806 04:57:24.114286 (+ 361us) client_negotiation.cc:240] Negotiation successful
0806 04:57:24.114906 (+ 620us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":681,"thread_start_us":460,"threads_started":1}
I20260806 04:57:24.138944 12347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:57:24.124739 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.96.66:35009 (local address 127.9.96.126:39063)
0806 04:57:24.124955 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0806 04:57:24.124963 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 04:57:24.125672 (+ 709us) server_negotiation.cc:407] Connection header received
0806 04:57:24.125903 (+ 231us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:57:24.125910 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:57:24.126011 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:57:24.126121 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:57:24.127627 (+ 1506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:57:24.128329 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:57:24.129752 (+ 1423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:57:24.130317 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:57:24.130465 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:57:24.132923 (+ 2458us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:57:24.132928 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:57:24.132931 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:57:24.132959 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:57:24.136365 (+ 3406us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:57:24.137016 (+ 651us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:57:24.137023 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:57:24.137026 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:57:24.137160 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:57:24.137516 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:57:24.137522 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:57:24.137525 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:57:24.137832 (+ 307us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:57:24.137984 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0806 04:57:24.138393 (+ 409us) server_negotiation.cc:299] Negotiation successful
0806 04:57:24.138604 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260806 04:57:24.139096 12668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:57:24.124757 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:39063 (local address 127.9.96.66:35009)
0806 04:57:24.125549 (+ 792us) negotiation.cc:107] Waiting for socket to connect
0806 04:57:24.125583 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0806 04:57:24.125760 (+ 177us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:57:24.126150 (+ 390us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:57:24.126158 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:57:24.126871 (+ 713us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:57:24.127485 (+ 614us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:57:24.127497 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:57:24.128473 (+ 976us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:57:24.128481 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:57:24.129612 (+ 1131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:57:24.129624 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:57:24.130507 (+ 883us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:57:24.130514 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:57:24.130706 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:57:24.131516 (+ 810us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:57:24.131533 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:57:24.132698 (+ 1165us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 04:57:24.136494 (+ 3796us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:57:24.136499 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:57:24.136503 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:57:24.136899 (+ 396us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:57:24.137275 (+ 376us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:57:24.137280 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:57:24.137284 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:57:24.137407 (+ 123us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:57:24.137953 (+ 546us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 04:57:24.137958 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 04:57:24.138137 (+ 179us) client_negotiation.cc:769] Sending connection context
0806 04:57:24.138304 (+ 167us) client_negotiation.cc:240] Negotiation successful
0806 04:57:24.138604 (+ 300us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":580,"thread_start_us":385,"threads_started":1}
I20260806 04:57:24.143831 12512 rpc_server.cc:307] RPC server started. Bound to: 127.9.96.66:40541
I20260806 04:57:24.143911 12710 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.96.66:40541 every 8 connection(s)
I20260806 04:57:24.145092 12512 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785992227050689-9601-0/minicluster-data/ts-1/data/info.pb
I20260806 04:57:24.147169 12712 heartbeater.cc:347] Connected to a master server at 127.9.96.126:39063
I20260806 04:57:24.147486 12712 heartbeater.cc:464] Registering TS with master...
I20260806 04:57:24.148125 12712 heartbeater.cc:511] Master 127.9.96.126:39063 requested a full tablet report, sending...
I20260806 04:57:24.149195 9601 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu as pid 12512
I20260806 04:57:24.149379 9601 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785992227050689-9601-0/minicluster-data/ts-1/wal/instance
I20260806 04:57:24.149621 12210 ts_manager.cc:194] Registered new tserver with Master: 61f45f84da5f4e89bd2a27ed0cdc051b (127.9.96.66:40541)
I20260806 04:57:24.149978 9601 external_mini_cluster.cc:1428] Setting key df2a9610f353b56f0c83375fd9acb5b8
I20260806 04:57:24.150694 12210 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.96.66@KRBTEST.COM'} at 127.9.96.66:57477
WARNING: no policy specified for kudu/127.9.96.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.96.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.96.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.67.keytab.
Entry for principal kudu/127.9.96.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.67.keytab.
I20260806 04:57:24.194855 9601 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
/tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785992227050689-9601-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785992227050689-9601-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785992227050689-9601-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785992227050689-9601-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.96.67:0
--local_ip_for_outbound_sockets=127.9.96.67
--webserver_interface=127.9.96.67
--webserver_port=0
--tserver_master_addrs=127.9.96.126:39063
--keytab_file=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.67.keytab
--principal=kudu/127.9.96.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.96.84:38505
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785992227050689-9601-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785992227050689-9601-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785992227050689-9601-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 04:57:24.511554 12720 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 04:57:24.511806 12720 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 04:57:24.511979 12720 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 04:57:24.519017 12720 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 04:57:24.519130 12720 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 04:57:24.519284 12720 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260806 04:57:24.519367 12720 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 04:57:24.519490 12720 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.96.67
I20260806 04:57:24.528786 12720 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.96.84:38505
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785992227050689-9601-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785992227050689-9601-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.67.keytab
--principal=kudu/127.9.96.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.96.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785992227050689-9601-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.96.67
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.9.96.126:39063
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.96.67
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785992227050689-9601-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 03:56:12 UTC on ccycloud.quasar-guijbm.root.comops.site
build id 4569618
ASAN enabled
I20260806 04:57:24.530074 12720 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 04:57:24.531411 12720 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 04:57:24.544229 12727 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:57:24.544286 12728 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:57:24.544308 12730 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:57:24.545439 12720 server_base.cc:1034] running on GCE node
I20260806 04:57:24.546007 12720 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 04:57:24.547003 12720 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 04:57:24.548223 12720 hybrid_clock.cc:648] HybridClock initialized: now 1785992244548185 us; error 35 us; skew 500 ppm
I20260806 04:57:24.548534 12720 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 04:57:24 dist-test-slave-x4qh krb5kdc[12131](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992244, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.96.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 04:57:24.551791 12720 init.cc:377] Logged in from keytab as kudu/127.9.96.67@KRBTEST.COM (short username kudu)
I20260806 04:57:24.553300 12720 webserver.cc:460] Webserver started at http://127.9.96.67:34689/ using document root <none> and password file <none>
I20260806 04:57:24.553944 12720 fs_manager.cc:359] Metadata directory not provided
I20260806 04:57:24.554093 12720 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 04:57:24.554448 12720 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 04:57:24.557222 12720 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785992227050689-9601-0/minicluster-data/ts-2/data/instance:
uuid: "7e334a6d68eb4d98853ce401bd25949b"
format_stamp: "Formatted at 2026-08-06 04:57:24 on dist-test-slave-x4qh"
server_key: "b821269430e2db7c03583b4bc10ce1cb"
server_key_iv: "5b6e5d64ff57a4e52cad9683d57f6d97"
server_key_version: "encryptionkey@0"
I20260806 04:57:24.557992 12720 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785992227050689-9601-0/minicluster-data/ts-2/wal/instance:
uuid: "7e334a6d68eb4d98853ce401bd25949b"
format_stamp: "Formatted at 2026-08-06 04:57:24 on dist-test-slave-x4qh"
server_key: "b821269430e2db7c03583b4bc10ce1cb"
server_key_iv: "5b6e5d64ff57a4e52cad9683d57f6d97"
server_key_version: "encryptionkey@0"
I20260806 04:57:24.562898 12720 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260806 04:57:24.566452 12743 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 04:57:24.567715 12720 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 04:57:24.567883 12720 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785992227050689-9601-0/minicluster-data/ts-2/data,/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785992227050689-9601-0/minicluster-data/ts-2/wal
uuid: "7e334a6d68eb4d98853ce401bd25949b"
format_stamp: "Formatted at 2026-08-06 04:57:24 on dist-test-slave-x4qh"
server_key: "b821269430e2db7c03583b4bc10ce1cb"
server_key_iv: "5b6e5d64ff57a4e52cad9683d57f6d97"
server_key_version: "encryptionkey@0"
I20260806 04:57:24.568079 12720 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785992227050689-9601-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785992227050689-9601-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785992227050689-9601-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 04:57:24.594440 12720 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 04:57:24.598435 12720 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 04:57:24.598743 12720 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 04:57:24.601629 12720 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 04:57:24.601764 12720 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 04:57:24.601884 12720 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 04:57:24.601981 12720 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 04:57:24 dist-test-slave-x4qh krb5kdc[12131](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992244, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.96.67@KRBTEST.COM for kudu/127.9.96.126@KRBTEST.COM
I20260806 04:57:24.621582 12507 heartbeater.cc:503] Master 127.9.96.126:39063 was elected leader, sending a full tablet report...
I20260806 04:57:24.627355 12347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:57:24.602506 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.96.67:33005 (local address 127.9.96.126:39063)
0806 04:57:24.602768 (+ 262us) server_negotiation.cc:206] Beginning negotiation
0806 04:57:24.602773 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 04:57:24.604222 (+ 1449us) server_negotiation.cc:407] Connection header received
0806 04:57:24.606401 (+ 2179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:57:24.606411 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:57:24.606509 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:57:24.606648 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:57:24.609074 (+ 2426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:57:24.610439 (+ 1365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:57:24.613645 (+ 3206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:57:24.614451 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:57:24.614634 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:57:24.620113 (+ 5479us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:57:24.620121 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:57:24.620126 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:57:24.620180 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:57:24.623815 (+ 3635us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:57:24.624720 (+ 905us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:57:24.624726 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:57:24.624730 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:57:24.624884 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:57:24.625406 (+ 522us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:57:24.625412 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:57:24.625415 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:57:24.625727 (+ 312us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:57:24.625901 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0806 04:57:24.626673 (+ 772us) server_negotiation.cc:299] Negotiation successful
0806 04:57:24.626937 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260806 04:57:24.628721 12753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:57:24.603081 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:39063 (local address 127.9.96.67:33005)
0806 04:57:24.604024 (+ 943us) negotiation.cc:107] Waiting for socket to connect
0806 04:57:24.604071 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0806 04:57:24.606067 (+ 1996us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:57:24.606922 (+ 855us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:57:24.606936 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:57:24.607888 (+ 952us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:57:24.608805 (+ 917us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:57:24.608858 (+ 53us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:57:24.610665 (+ 1807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:57:24.610674 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:57:24.613420 (+ 2746us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:57:24.613437 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:57:24.614740 (+ 1303us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:57:24.614748 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:57:24.615018 (+ 270us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:57:24.615937 (+ 919us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:57:24.615974 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:57:24.619832 (+ 3858us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 04:57:24.624003 (+ 4171us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:57:24.624014 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:57:24.624028 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:57:24.624548 (+ 520us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:57:24.625020 (+ 472us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:57:24.625026 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:57:24.625030 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:57:24.625265 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:57:24.625873 (+ 608us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 04:57:24.625882 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 04:57:24.626221 (+ 339us) client_negotiation.cc:769] Sending connection context
0806 04:57:24.626503 (+ 282us) client_negotiation.cc:240] Negotiation successful
0806 04:57:24.627023 (+ 520us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":686,"thread_start_us":430,"threads_started":1}
I20260806 04:57:24.653260 12835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:57:24.635571 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:39063 (local address 127.9.96.67:48575)
0806 04:57:24.636399 (+ 828us) negotiation.cc:107] Waiting for socket to connect
0806 04:57:24.636416 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 04:57:24.636586 (+ 170us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:57:24.637017 (+ 431us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:57:24.637021 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:57:24.637706 (+ 685us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:57:24.638253 (+ 547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:57:24.638279 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:57:24.639686 (+ 1407us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:57:24.639694 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:57:24.641370 (+ 1676us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:57:24.641395 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:57:24.642748 (+ 1353us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:57:24.642754 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:57:24.642927 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:57:24.643801 (+ 874us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:57:24.643839 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:57:24.645151 (+ 1312us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 04:57:24.649729 (+ 4578us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:57:24.649738 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:57:24.649745 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:57:24.650497 (+ 752us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:57:24.650994 (+ 497us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:57:24.651001 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:57:24.651007 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:57:24.651206 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:57:24.652180 (+ 974us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 04:57:24.652184 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 04:57:24.652362 (+ 178us) client_negotiation.cc:769] Sending connection context
0806 04:57:24.652564 (+ 202us) client_negotiation.cc:240] Negotiation successful
0806 04:57:24.652879 (+ 315us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":687,"thread_start_us":513,"threads_started":1}
I20260806 04:57:24.653740 12347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:57:24.635472 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.96.67:48575 (local address 127.9.96.126:39063)
0806 04:57:24.635674 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0806 04:57:24.635682 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 04:57:24.636510 (+ 828us) server_negotiation.cc:407] Connection header received
0806 04:57:24.636767 (+ 257us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:57:24.636774 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:57:24.636865 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:57:24.636979 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:57:24.638415 (+ 1436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:57:24.639490 (+ 1075us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:57:24.641551 (+ 2061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:57:24.642528 (+ 977us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:57:24.642698 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:57:24.645471 (+ 2773us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:57:24.645478 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:57:24.645484 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:57:24.645530 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:57:24.649516 (+ 3986us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:57:24.650670 (+ 1154us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:57:24.650676 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:57:24.650682 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:57:24.650846 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:57:24.651433 (+ 587us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:57:24.651441 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:57:24.651447 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:57:24.651985 (+ 538us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:57:24.652189 (+ 204us) server_negotiation.cc:1035] Waiting for connection context
0806 04:57:24.652716 (+ 527us) server_negotiation.cc:299] Negotiation successful
0806 04:57:24.653032 (+ 316us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260806 04:57:24.662822 12720 rpc_server.cc:307] RPC server started. Bound to: 127.9.96.67:34229
I20260806 04:57:24.662902 12909 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.96.67:34229 every 8 connection(s)
I20260806 04:57:24.664072 12720 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785992227050689-9601-0/minicluster-data/ts-2/data/info.pb
I20260806 04:57:24.666124 12910 heartbeater.cc:347] Connected to a master server at 127.9.96.126:39063
I20260806 04:57:24.666371 12910 heartbeater.cc:464] Registering TS with master...
I20260806 04:57:24.667146 12910 heartbeater.cc:511] Master 127.9.96.126:39063 requested a full tablet report, sending...
I20260806 04:57:24.668505 12208 ts_manager.cc:194] Registered new tserver with Master: 7e334a6d68eb4d98853ce401bd25949b (127.9.96.67:34229)
I20260806 04:57:24.669312 12208 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.96.67@KRBTEST.COM'} at 127.9.96.67:33005
I20260806 04:57:24.672487 9601 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu as pid 12720
I20260806 04:57:24.672665 9601 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785992227050689-9601-0/minicluster-data/ts-2/wal/instance
I20260806 04:57:24.673105 9601 external_mini_cluster.cc:1428] Setting key 920b0cbe1ac8f15629721161eb26cbe1
I20260806 04:57:24.675907 9601 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 06 04:57:24 dist-test-slave-x4qh krb5kdc[12131](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992244, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 06 04:57:24 dist-test-slave-x4qh krb5kdc[12131](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992244, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.96.126@KRBTEST.COM
I20260806 04:57:24.723662 12347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:57:24.710143 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47864 (local address 127.9.96.126:39063)
0806 04:57:24.710398 (+ 255us) server_negotiation.cc:206] Beginning negotiation
0806 04:57:24.710405 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 04:57:24.710635 (+ 230us) server_negotiation.cc:407] Connection header received
0806 04:57:24.710778 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:57:24.710782 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:57:24.710848 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:57:24.710935 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:57:24.712411 (+ 1476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:57:24.713132 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:57:24.714286 (+ 1154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:57:24.714672 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:57:24.714762 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:57:24.718861 (+ 4099us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:57:24.718867 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:57:24.718870 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:57:24.718903 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:57:24.721353 (+ 2450us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:57:24.721935 (+ 582us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:57:24.721939 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:57:24.721941 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:57:24.722055 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:57:24.722395 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:57:24.722398 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:57:24.722400 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:57:24.722650 (+ 250us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:57:24.722824 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0806 04:57:24.723062 (+ 238us) server_negotiation.cc:299] Negotiation successful
0806 04:57:24.723216 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260806 04:57:24.727816 12208 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47864:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 04:57:24.734863 12208 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 04:57:24.760746 12930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:57:24.747083 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.66:40541 (local address 127.0.0.1:44726)
0806 04:57:24.748372 (+ 1289us) negotiation.cc:107] Waiting for socket to connect
0806 04:57:24.748409 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0806 04:57:24.748562 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:57:24.750815 (+ 2253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:57:24.750823 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:57:24.750866 (+ 43us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 04:57:24.751502 (+ 636us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:57:24.751521 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:57:24.754276 (+ 2755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:57:24.754284 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:57:24.757502 (+ 3218us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:57:24.757524 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:57:24.759600 (+ 2076us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:57:24.759607 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:57:24.759941 (+ 334us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:57:24.760000 (+ 59us) client_negotiation.cc:769] Sending connection context
0806 04:57:24.760127 (+ 127us) client_negotiation.cc:240] Negotiation successful
0806 04:57:24.760321 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1027,"mutex_wait_us":16,"thread_start_us":2336,"threads_started":1}
I20260806 04:57:24.760964 12929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:57:24.747269 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44726 (local address 127.9.96.66:40541)
0806 04:57:24.749998 (+ 2729us) server_negotiation.cc:206] Beginning negotiation
0806 04:57:24.750023 (+ 25us) server_negotiation.cc:399] Waiting for connection header
0806 04:57:24.750079 (+ 56us) server_negotiation.cc:407] Connection header received
0806 04:57:24.750198 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:57:24.750209 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:57:24.750516 (+ 307us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:57:24.751777 (+ 1261us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 04:57:24.752017 (+ 240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:57:24.754082 (+ 2065us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:57:24.757713 (+ 3631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:57:24.759246 (+ 1533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:57:24.759946 (+ 700us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:57:24.760282 (+ 336us) server_negotiation.cc:1035] Waiting for connection context
0806 04:57:24.760447 (+ 165us) server_negotiation.cc:299] Negotiation successful
0806 04:57:24.760675 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2568,"thread_start_us":2494,"threads_started":1}
I20260806 04:57:24.764492 12931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:57:24.748035 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60914 (local address 127.9.96.67:34229)
0806 04:57:24.749250 (+ 1215us) server_negotiation.cc:206] Beginning negotiation
0806 04:57:24.749258 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 04:57:24.751425 (+ 2167us) server_negotiation.cc:407] Connection header received
0806 04:57:24.751551 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:57:24.751562 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:57:24.751917 (+ 355us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:57:24.752121 (+ 204us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 04:57:24.754678 (+ 2557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:57:24.756915 (+ 2237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:57:24.761483 (+ 4568us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:57:24.763024 (+ 1541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:57:24.763117 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:57:24.763359 (+ 242us) server_negotiation.cc:1035] Waiting for connection context
0806 04:57:24.763993 (+ 634us) server_negotiation.cc:299] Negotiation successful
0806 04:57:24.764160 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1025,"thread_start_us":832,"threads_started":1}
I20260806 04:57:24.765295 12928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:57:24.747449 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.67:34229 (local address 127.0.0.1:60914)
0806 04:57:24.751056 (+ 3607us) negotiation.cc:107] Waiting for socket to connect
0806 04:57:24.751080 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0806 04:57:24.751241 (+ 161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:57:24.752558 (+ 1317us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:57:24.752565 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:57:24.752601 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 04:57:24.753174 (+ 573us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:57:24.753190 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:57:24.757126 (+ 3936us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:57:24.757134 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:57:24.761288 (+ 4154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:57:24.761304 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:57:24.763601 (+ 2297us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:57:24.763609 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:57:24.763788 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:57:24.763819 (+ 31us) client_negotiation.cc:769] Sending connection context
0806 04:57:24.764688 (+ 869us) client_negotiation.cc:240] Negotiation successful
0806 04:57:24.764836 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3402,"mutex_wait_us":462,"thread_start_us":510,"threads_started":1}
I20260806 04:57:24.765599 12609 tablet_service.cc:1467] Processing CreateTablet for tablet f3288924f5d24b46b6924e00565f996a (DEFAULT_TABLE table=test-table [id=6a3703d849064091a2fdaf55a08cb080]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 04:57:24.767081 12609 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f3288924f5d24b46b6924e00565f996a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 04:57:24.769068 12933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:57:24.749286 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.65:41697 (local address 127.0.0.1:50012)
0806 04:57:24.761288 (+ 12002us) negotiation.cc:107] Waiting for socket to connect
0806 04:57:24.761306 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 04:57:24.761425 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:57:24.762327 (+ 902us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:57:24.762332 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:57:24.762356 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 04:57:24.762692 (+ 336us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:57:24.762704 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:57:24.764821 (+ 2117us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:57:24.764826 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:57:24.766824 (+ 1998us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:57:24.766837 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:57:24.768295 (+ 1458us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:57:24.768301 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:57:24.768474 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:57:24.768510 (+ 36us) client_negotiation.cc:769] Sending connection context
0806 04:57:24.768627 (+ 117us) client_negotiation.cc:240] Negotiation successful
0806 04:57:24.768746 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11877,"thread_start_us":11745,"threads_started":1}
I20260806 04:57:24.769104 12934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:57:24.749589 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50012 (local address 127.9.96.65:41697)
0806 04:57:24.760817 (+ 11228us) server_negotiation.cc:206] Beginning negotiation
0806 04:57:24.760828 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 04:57:24.761550 (+ 722us) server_negotiation.cc:407] Connection header received
0806 04:57:24.761682 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:57:24.761691 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:57:24.761926 (+ 235us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:57:24.762116 (+ 190us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 04:57:24.763099 (+ 983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:57:24.764627 (+ 1528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:57:24.766982 (+ 2355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:57:24.768093 (+ 1111us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:57:24.768249 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:57:24.768459 (+ 210us) server_negotiation.cc:1035] Waiting for connection context
0806 04:57:24.768670 (+ 211us) server_negotiation.cc:299] Negotiation successful
0806 04:57:24.768826 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11016,"thread_start_us":7963,"threads_started":1}
I20260806 04:57:24.769016 12808 tablet_service.cc:1467] Processing CreateTablet for tablet f3288924f5d24b46b6924e00565f996a (DEFAULT_TABLE table=test-table [id=6a3703d849064091a2fdaf55a08cb080]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 04:57:24.770464 12808 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f3288924f5d24b46b6924e00565f996a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 04:57:24.774624 12408 tablet_service.cc:1467] Processing CreateTablet for tablet f3288924f5d24b46b6924e00565f996a (DEFAULT_TABLE table=test-table [id=6a3703d849064091a2fdaf55a08cb080]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 04:57:24.775959 12408 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f3288924f5d24b46b6924e00565f996a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 04:57:24.782021 12936 tablet_bootstrap.cc:492] T f3288924f5d24b46b6924e00565f996a P 61f45f84da5f4e89bd2a27ed0cdc051b: Bootstrap starting.
I20260806 04:57:24.785669 12936 tablet_bootstrap.cc:654] T f3288924f5d24b46b6924e00565f996a P 61f45f84da5f4e89bd2a27ed0cdc051b: Neither blocks nor log segments found. Creating new log.
I20260806 04:57:24.786434 12937 tablet_bootstrap.cc:492] T f3288924f5d24b46b6924e00565f996a P 7e334a6d68eb4d98853ce401bd25949b: Bootstrap starting.
I20260806 04:57:24.787400 12936 log.cc:824] T f3288924f5d24b46b6924e00565f996a P 61f45f84da5f4e89bd2a27ed0cdc051b: Log is configured to *not* fsync() on all Append() calls
I20260806 04:57:24.790051 12937 tablet_bootstrap.cc:654] T f3288924f5d24b46b6924e00565f996a P 7e334a6d68eb4d98853ce401bd25949b: Neither blocks nor log segments found. Creating new log.
I20260806 04:57:24.790788 12939 tablet_bootstrap.cc:492] T f3288924f5d24b46b6924e00565f996a P 4b26fa9b4a984025a159d0f38be81428: Bootstrap starting.
I20260806 04:57:24.791363 12937 log.cc:824] T f3288924f5d24b46b6924e00565f996a P 7e334a6d68eb4d98853ce401bd25949b: Log is configured to *not* fsync() on all Append() calls
I20260806 04:57:24.791579 12936 tablet_bootstrap.cc:492] T f3288924f5d24b46b6924e00565f996a P 61f45f84da5f4e89bd2a27ed0cdc051b: No bootstrap required, opened a new log
I20260806 04:57:24.791857 12936 ts_tablet_manager.cc:1397] T f3288924f5d24b46b6924e00565f996a P 61f45f84da5f4e89bd2a27ed0cdc051b: Time spent bootstrapping tablet: real 0.010s user 0.005s sys 0.003s
I20260806 04:57:24.793931 12937 tablet_bootstrap.cc:492] T f3288924f5d24b46b6924e00565f996a P 7e334a6d68eb4d98853ce401bd25949b: No bootstrap required, opened a new log
I20260806 04:57:24.794195 12937 ts_tablet_manager.cc:1397] T f3288924f5d24b46b6924e00565f996a P 7e334a6d68eb4d98853ce401bd25949b: Time spent bootstrapping tablet: real 0.008s user 0.003s sys 0.003s
I20260806 04:57:24.794638 12939 tablet_bootstrap.cc:654] T f3288924f5d24b46b6924e00565f996a P 4b26fa9b4a984025a159d0f38be81428: Neither blocks nor log segments found. Creating new log.
I20260806 04:57:24.795847 12939 log.cc:824] T f3288924f5d24b46b6924e00565f996a P 4b26fa9b4a984025a159d0f38be81428: Log is configured to *not* fsync() on all Append() calls
I20260806 04:57:24.798664 12939 tablet_bootstrap.cc:492] T f3288924f5d24b46b6924e00565f996a P 4b26fa9b4a984025a159d0f38be81428: No bootstrap required, opened a new log
I20260806 04:57:24.798908 12939 ts_tablet_manager.cc:1397] T f3288924f5d24b46b6924e00565f996a P 4b26fa9b4a984025a159d0f38be81428: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260806 04:57:24.801997 12937 raft_consensus.cc:348] T f3288924f5d24b46b6924e00565f996a P 7e334a6d68eb4d98853ce401bd25949b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61f45f84da5f4e89bd2a27ed0cdc051b" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 40541 } } peers { permanent_uuid: "7e334a6d68eb4d98853ce401bd25949b" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 34229 } } peers { permanent_uuid: "4b26fa9b4a984025a159d0f38be81428" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 41697 } }
I20260806 04:57:24.802362 12937 raft_consensus.cc:374] T f3288924f5d24b46b6924e00565f996a P 7e334a6d68eb4d98853ce401bd25949b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 04:57:24.802187 12936 raft_consensus.cc:348] T f3288924f5d24b46b6924e00565f996a P 61f45f84da5f4e89bd2a27ed0cdc051b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61f45f84da5f4e89bd2a27ed0cdc051b" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 40541 } } peers { permanent_uuid: "7e334a6d68eb4d98853ce401bd25949b" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 34229 } } peers { permanent_uuid: "4b26fa9b4a984025a159d0f38be81428" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 41697 } }
I20260806 04:57:24.802492 12937 raft_consensus.cc:729] T f3288924f5d24b46b6924e00565f996a P 7e334a6d68eb4d98853ce401bd25949b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e334a6d68eb4d98853ce401bd25949b, State: Initialized, Role: FOLLOWER
I20260806 04:57:24.802605 12936 raft_consensus.cc:374] T f3288924f5d24b46b6924e00565f996a P 61f45f84da5f4e89bd2a27ed0cdc051b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 04:57:24.802704 12936 raft_consensus.cc:729] T f3288924f5d24b46b6924e00565f996a P 61f45f84da5f4e89bd2a27ed0cdc051b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 61f45f84da5f4e89bd2a27ed0cdc051b, State: Initialized, Role: FOLLOWER
I20260806 04:57:24.803172 12937 consensus_queue.cc:260] T f3288924f5d24b46b6924e00565f996a P 7e334a6d68eb4d98853ce401bd25949b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61f45f84da5f4e89bd2a27ed0cdc051b" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 40541 } } peers { permanent_uuid: "7e334a6d68eb4d98853ce401bd25949b" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 34229 } } peers { permanent_uuid: "4b26fa9b4a984025a159d0f38be81428" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 41697 } }
I20260806 04:57:24.803287 12936 consensus_queue.cc:260] T f3288924f5d24b46b6924e00565f996a P 61f45f84da5f4e89bd2a27ed0cdc051b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61f45f84da5f4e89bd2a27ed0cdc051b" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 40541 } } peers { permanent_uuid: "7e334a6d68eb4d98853ce401bd25949b" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 34229 } } peers { permanent_uuid: "4b26fa9b4a984025a159d0f38be81428" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 41697 } }
I20260806 04:57:24.804582 12712 heartbeater.cc:503] Master 127.9.96.126:39063 was elected leader, sending a full tablet report...
I20260806 04:57:24.805073 12936 ts_tablet_manager.cc:1428] T f3288924f5d24b46b6924e00565f996a P 61f45f84da5f4e89bd2a27ed0cdc051b: Time spent starting tablet: real 0.013s user 0.012s sys 0.000s
I20260806 04:57:24.805932 12910 heartbeater.cc:503] Master 127.9.96.126:39063 was elected leader, sending a full tablet report...
I20260806 04:57:24.807121 12937 ts_tablet_manager.cc:1428] T f3288924f5d24b46b6924e00565f996a P 7e334a6d68eb4d98853ce401bd25949b: Time spent starting tablet: real 0.013s user 0.013s sys 0.001s
I20260806 04:57:24.808820 12939 raft_consensus.cc:348] T f3288924f5d24b46b6924e00565f996a P 4b26fa9b4a984025a159d0f38be81428 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61f45f84da5f4e89bd2a27ed0cdc051b" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 40541 } } peers { permanent_uuid: "7e334a6d68eb4d98853ce401bd25949b" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 34229 } } peers { permanent_uuid: "4b26fa9b4a984025a159d0f38be81428" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 41697 } }
I20260806 04:57:24.809202 12939 raft_consensus.cc:374] T f3288924f5d24b46b6924e00565f996a P 4b26fa9b4a984025a159d0f38be81428 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 04:57:24.809322 12939 raft_consensus.cc:729] T f3288924f5d24b46b6924e00565f996a P 4b26fa9b4a984025a159d0f38be81428 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b26fa9b4a984025a159d0f38be81428, State: Initialized, Role: FOLLOWER
I20260806 04:57:24.809927 12939 consensus_queue.cc:260] T f3288924f5d24b46b6924e00565f996a P 4b26fa9b4a984025a159d0f38be81428 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61f45f84da5f4e89bd2a27ed0cdc051b" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 40541 } } peers { permanent_uuid: "7e334a6d68eb4d98853ce401bd25949b" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 34229 } } peers { permanent_uuid: "4b26fa9b4a984025a159d0f38be81428" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 41697 } }
I20260806 04:57:24.811889 12939 ts_tablet_manager.cc:1428] T f3288924f5d24b46b6924e00565f996a P 4b26fa9b4a984025a159d0f38be81428: Time spent starting tablet: real 0.013s user 0.014s sys 0.000s
W20260806 04:57:24.865304 12508 tablet.cc:2369] T f3288924f5d24b46b6924e00565f996a P 4b26fa9b4a984025a159d0f38be81428: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 04:57:24.887238 12944 raft_consensus.cc:482] T f3288924f5d24b46b6924e00565f996a P 61f45f84da5f4e89bd2a27ed0cdc051b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 04:57:24.887646 12944 raft_consensus.cc:504] T f3288924f5d24b46b6924e00565f996a P 61f45f84da5f4e89bd2a27ed0cdc051b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61f45f84da5f4e89bd2a27ed0cdc051b" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 40541 } } peers { permanent_uuid: "7e334a6d68eb4d98853ce401bd25949b" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 34229 } } peers { permanent_uuid: "4b26fa9b4a984025a159d0f38be81428" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 41697 } }
I20260806 04:57:24.889277 12944 leader_election.cc:288] T f3288924f5d24b46b6924e00565f996a P 61f45f84da5f4e89bd2a27ed0cdc051b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e334a6d68eb4d98853ce401bd25949b (127.9.96.67:34229), 4b26fa9b4a984025a159d0f38be81428 (127.9.96.65:41697)
I20260806 04:57:24.896310 12931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:57:24.889744 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.96.66:52179 (local address 127.9.96.67:34229)
0806 04:57:24.889929 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0806 04:57:24.889933 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 04:57:24.890460 (+ 527us) server_negotiation.cc:407] Connection header received
0806 04:57:24.890732 (+ 272us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:57:24.890739 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:57:24.890825 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:57:24.890894 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 04:57:24.891641 (+ 747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:57:24.892697 (+ 1056us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:57:24.894515 (+ 1818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:57:24.895324 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:57:24.895491 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:57:24.895613 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0806 04:57:24.895901 (+ 288us) server_negotiation.cc:299] Negotiation successful
0806 04:57:24.896029 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260806 04:57:24.896498 12934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:57:24.890719 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.96.66:58897 (local address 127.9.96.65:41697)
0806 04:57:24.890922 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0806 04:57:24.890929 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 04:57:24.891125 (+ 196us) server_negotiation.cc:407] Connection header received
0806 04:57:24.891438 (+ 313us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:57:24.891447 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:57:24.891514 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:57:24.891621 (+ 107us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 04:57:24.892065 (+ 444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:57:24.893016 (+ 951us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:57:24.894714 (+ 1698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:57:24.895563 (+ 849us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:57:24.895715 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:57:24.895822 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0806 04:57:24.896110 (+ 288us) server_negotiation.cc:299] Negotiation successful
0806 04:57:24.896235 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260806 04:57:24.896436 12946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:57:24.889654 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.67:34229 (local address 127.9.96.66:52179)
0806 04:57:24.890345 (+ 691us) negotiation.cc:107] Waiting for socket to connect
0806 04:57:24.890364 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0806 04:57:24.890510 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:57:24.891080 (+ 570us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:57:24.891083 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:57:24.891110 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 04:57:24.891492 (+ 382us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:57:24.891501 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:57:24.892836 (+ 1335us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:57:24.892839 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:57:24.894359 (+ 1520us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:57:24.894371 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:57:24.895569 (+ 1198us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:57:24.895574 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:57:24.895737 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:57:24.895763 (+ 26us) client_negotiation.cc:769] Sending connection context
0806 04:57:24.895875 (+ 112us) client_negotiation.cc:240] Negotiation successful
0806 04:57:24.896003 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":522,"mutex_wait_us":54,"thread_start_us":354,"threads_started":1}
I20260806 04:57:24.896628 12947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:57:24.890428 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.65:41697 (local address 127.9.96.66:58897)
0806 04:57:24.891023 (+ 595us) negotiation.cc:107] Waiting for socket to connect
0806 04:57:24.891037 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 04:57:24.891272 (+ 235us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:57:24.891627 (+ 355us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:57:24.891629 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:57:24.891647 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 04:57:24.891911 (+ 264us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:57:24.891917 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:57:24.893157 (+ 1240us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:57:24.893162 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:57:24.894554 (+ 1392us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:57:24.894566 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:57:24.895783 (+ 1217us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:57:24.895789 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:57:24.895953 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:57:24.895977 (+ 24us) client_negotiation.cc:769] Sending connection context
0806 04:57:24.896083 (+ 106us) client_negotiation.cc:240] Negotiation successful
0806 04:57:24.896207 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":475,"spinlock_wait_cycles":36992,"thread_start_us":298,"threads_started":1}
W20260806 04:57:24.897363 12713 tablet.cc:2369] T f3288924f5d24b46b6924e00565f996a P 61f45f84da5f4e89bd2a27ed0cdc051b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 04:57:24.897853 12845 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f3288924f5d24b46b6924e00565f996a" candidate_uuid: "61f45f84da5f4e89bd2a27ed0cdc051b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e334a6d68eb4d98853ce401bd25949b" is_pre_election: true
I20260806 04:57:24.897908 12441 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f3288924f5d24b46b6924e00565f996a" candidate_uuid: "61f45f84da5f4e89bd2a27ed0cdc051b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b26fa9b4a984025a159d0f38be81428" is_pre_election: true
I20260806 04:57:24.898382 12845 raft_consensus.cc:2454] T f3288924f5d24b46b6924e00565f996a P 7e334a6d68eb4d98853ce401bd25949b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 61f45f84da5f4e89bd2a27ed0cdc051b in term 0.
I20260806 04:57:24.898504 12441 raft_consensus.cc:2454] T f3288924f5d24b46b6924e00565f996a P 4b26fa9b4a984025a159d0f38be81428 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 61f45f84da5f4e89bd2a27ed0cdc051b in term 0.
I20260806 04:57:24.899215 12544 leader_election.cc:302] T f3288924f5d24b46b6924e00565f996a P 61f45f84da5f4e89bd2a27ed0cdc051b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 61f45f84da5f4e89bd2a27ed0cdc051b, 7e334a6d68eb4d98853ce401bd25949b; no voters:
I20260806 04:57:24.899631 12944 raft_consensus.cc:2781] T f3288924f5d24b46b6924e00565f996a P 61f45f84da5f4e89bd2a27ed0cdc051b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 04:57:24.899794 12944 raft_consensus.cc:482] T f3288924f5d24b46b6924e00565f996a P 61f45f84da5f4e89bd2a27ed0cdc051b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 04:57:24.899914 12944 raft_consensus.cc:3037] T f3288924f5d24b46b6924e00565f996a P 61f45f84da5f4e89bd2a27ed0cdc051b [term 0 FOLLOWER]: Advancing to term 1
I20260806 04:57:24.902338 12944 raft_consensus.cc:504] T f3288924f5d24b46b6924e00565f996a P 61f45f84da5f4e89bd2a27ed0cdc051b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61f45f84da5f4e89bd2a27ed0cdc051b" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 40541 } } peers { permanent_uuid: "7e334a6d68eb4d98853ce401bd25949b" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 34229 } } peers { permanent_uuid: "4b26fa9b4a984025a159d0f38be81428" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 41697 } }
I20260806 04:57:24.902911 12944 leader_election.cc:288] T f3288924f5d24b46b6924e00565f996a P 61f45f84da5f4e89bd2a27ed0cdc051b [CANDIDATE]: Term 1 election: Requested vote from peers 7e334a6d68eb4d98853ce401bd25949b (127.9.96.67:34229), 4b26fa9b4a984025a159d0f38be81428 (127.9.96.65:41697)
I20260806 04:57:24.903357 12845 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f3288924f5d24b46b6924e00565f996a" candidate_uuid: "61f45f84da5f4e89bd2a27ed0cdc051b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e334a6d68eb4d98853ce401bd25949b"
I20260806 04:57:24.903467 12441 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f3288924f5d24b46b6924e00565f996a" candidate_uuid: "61f45f84da5f4e89bd2a27ed0cdc051b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b26fa9b4a984025a159d0f38be81428"
I20260806 04:57:24.903631 12845 raft_consensus.cc:3037] T f3288924f5d24b46b6924e00565f996a P 7e334a6d68eb4d98853ce401bd25949b [term 0 FOLLOWER]: Advancing to term 1
I20260806 04:57:24.903657 12441 raft_consensus.cc:3037] T f3288924f5d24b46b6924e00565f996a P 4b26fa9b4a984025a159d0f38be81428 [term 0 FOLLOWER]: Advancing to term 1
I20260806 04:57:24.905827 12441 raft_consensus.cc:2454] T f3288924f5d24b46b6924e00565f996a P 4b26fa9b4a984025a159d0f38be81428 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 61f45f84da5f4e89bd2a27ed0cdc051b in term 1.
I20260806 04:57:24.905838 12845 raft_consensus.cc:2454] T f3288924f5d24b46b6924e00565f996a P 7e334a6d68eb4d98853ce401bd25949b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 61f45f84da5f4e89bd2a27ed0cdc051b in term 1.
I20260806 04:57:24.906212 12544 leader_election.cc:302] T f3288924f5d24b46b6924e00565f996a P 61f45f84da5f4e89bd2a27ed0cdc051b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4b26fa9b4a984025a159d0f38be81428, 61f45f84da5f4e89bd2a27ed0cdc051b; no voters:
I20260806 04:57:24.906539 12944 raft_consensus.cc:2781] T f3288924f5d24b46b6924e00565f996a P 61f45f84da5f4e89bd2a27ed0cdc051b [term 1 FOLLOWER]: Leader election won for term 1
I20260806 04:57:24.907028 12944 raft_consensus.cc:686] T f3288924f5d24b46b6924e00565f996a P 61f45f84da5f4e89bd2a27ed0cdc051b [term 1 LEADER]: Becoming Leader. State: Replica: 61f45f84da5f4e89bd2a27ed0cdc051b, State: Running, Role: LEADER
I20260806 04:57:24.907589 12944 consensus_queue.cc:237] T f3288924f5d24b46b6924e00565f996a P 61f45f84da5f4e89bd2a27ed0cdc051b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61f45f84da5f4e89bd2a27ed0cdc051b" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 40541 } } peers { permanent_uuid: "7e334a6d68eb4d98853ce401bd25949b" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 34229 } } peers { permanent_uuid: "4b26fa9b4a984025a159d0f38be81428" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 41697 } }
I20260806 04:57:24.912189 12208 catalog_manager.cc:5495] T f3288924f5d24b46b6924e00565f996a P 61f45f84da5f4e89bd2a27ed0cdc051b reported cstate change: term changed from 0 to 1, leader changed from <none> to 61f45f84da5f4e89bd2a27ed0cdc051b (127.9.96.66). New cstate: current_term: 1 leader_uuid: "61f45f84da5f4e89bd2a27ed0cdc051b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61f45f84da5f4e89bd2a27ed0cdc051b" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 40541 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7e334a6d68eb4d98853ce401bd25949b" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 34229 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4b26fa9b4a984025a159d0f38be81428" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 41697 } health_report { overall_health: UNKNOWN } } }
W20260806 04:57:24.915740 12911 tablet.cc:2369] T f3288924f5d24b46b6924e00565f996a P 7e334a6d68eb4d98853ce401bd25949b: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 06 04:57:24 dist-test-slave-x4qh krb5kdc[12131](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992242, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.96.126@KRBTEST.COM for kudu/127.9.96.126@KRBTEST.COM
I20260806 04:57:24.975385 12347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:57:24.961638 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47866 (local address 127.9.96.126:39063)
0806 04:57:24.961863 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0806 04:57:24.961871 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 04:57:24.962519 (+ 648us) server_negotiation.cc:407] Connection header received
0806 04:57:24.962627 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:57:24.962631 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:57:24.962692 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:57:24.962808 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:57:24.964037 (+ 1229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:57:24.964923 (+ 886us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:57:24.966226 (+ 1303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:57:24.966730 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:57:24.966900 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:57:24.970075 (+ 3175us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:57:24.970080 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:57:24.970083 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:57:24.970111 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:57:24.972619 (+ 2508us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:57:24.973342 (+ 723us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:57:24.973349 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:57:24.973356 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:57:24.973488 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:57:24.973850 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:57:24.973858 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:57:24.973861 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:57:24.974114 (+ 253us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:57:24.974276 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0806 04:57:24.974672 (+ 396us) server_negotiation.cc:299] Negotiation successful
0806 04:57:24.974995 (+ 323us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":119,"server-negotiator.queue_time_us":70}
I20260806 04:57:24.975423 12955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:57:24.961756 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:39063 (local address 127.0.0.1:47866)
0806 04:57:24.962387 (+ 631us) negotiation.cc:107] Waiting for socket to connect
0806 04:57:24.962407 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0806 04:57:24.962527 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:57:24.962846 (+ 319us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:57:24.962849 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:57:24.963608 (+ 759us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:57:24.963895 (+ 287us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:57:24.963905 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:57:24.965040 (+ 1135us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:57:24.965043 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:57:24.966070 (+ 1027us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:57:24.966082 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:57:24.966922 (+ 840us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:57:24.966930 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:57:24.967076 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:57:24.967730 (+ 654us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:57:24.967752 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:57:24.969899 (+ 2147us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 04:57:24.972763 (+ 2864us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:57:24.972777 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:57:24.972785 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:57:24.973223 (+ 438us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:57:24.973595 (+ 372us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:57:24.973601 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:57:24.973605 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:57:24.973745 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:57:24.974251 (+ 506us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 04:57:24.974259 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 04:57:24.974423 (+ 164us) client_negotiation.cc:769] Sending connection context
0806 04:57:24.974586 (+ 163us) client_negotiation.cc:240] Negotiation successful
0806 04:57:24.974867 (+ 281us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":481,"thread_start_us":348,"threads_started":1}
I20260806 04:57:24.980324 12208 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.9.96.126@KRBTEST.COM'} at 127.0.0.1:47866:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260806 04:57:24.981324 12208 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 04:57:24.988420 12609 tablet_service.cc:1467] Processing CreateTablet for tablet 5474259a59a64b3ea875c6eca0e2eebb (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=b91b0f4160e0411ba6f0b683ef399814]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 04:57:24.988211 12408 tablet_service.cc:1467] Processing CreateTablet for tablet 5474259a59a64b3ea875c6eca0e2eebb (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=b91b0f4160e0411ba6f0b683ef399814]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 04:57:24.989243 12609 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5474259a59a64b3ea875c6eca0e2eebb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 04:57:24.989243 12408 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5474259a59a64b3ea875c6eca0e2eebb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 04:57:24.988952 12808 tablet_service.cc:1467] Processing CreateTablet for tablet 5474259a59a64b3ea875c6eca0e2eebb (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=b91b0f4160e0411ba6f0b683ef399814]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 04:57:24.989797 12808 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5474259a59a64b3ea875c6eca0e2eebb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 04:57:24.994982 12937 tablet_bootstrap.cc:492] T 5474259a59a64b3ea875c6eca0e2eebb P 7e334a6d68eb4d98853ce401bd25949b: Bootstrap starting.
I20260806 04:57:24.996518 12936 tablet_bootstrap.cc:492] T 5474259a59a64b3ea875c6eca0e2eebb P 61f45f84da5f4e89bd2a27ed0cdc051b: Bootstrap starting.
I20260806 04:57:24.996768 12937 tablet_bootstrap.cc:654] T 5474259a59a64b3ea875c6eca0e2eebb P 7e334a6d68eb4d98853ce401bd25949b: Neither blocks nor log segments found. Creating new log.
I20260806 04:57:24.998082 12939 tablet_bootstrap.cc:492] T 5474259a59a64b3ea875c6eca0e2eebb P 4b26fa9b4a984025a159d0f38be81428: Bootstrap starting.
I20260806 04:57:24.998389 12936 tablet_bootstrap.cc:654] T 5474259a59a64b3ea875c6eca0e2eebb P 61f45f84da5f4e89bd2a27ed0cdc051b: Neither blocks nor log segments found. Creating new log.
I20260806 04:57:24.999889 12939 tablet_bootstrap.cc:654] T 5474259a59a64b3ea875c6eca0e2eebb P 4b26fa9b4a984025a159d0f38be81428: Neither blocks nor log segments found. Creating new log.
I20260806 04:57:25.000300 12937 tablet_bootstrap.cc:492] T 5474259a59a64b3ea875c6eca0e2eebb P 7e334a6d68eb4d98853ce401bd25949b: No bootstrap required, opened a new log
I20260806 04:57:25.000448 12937 ts_tablet_manager.cc:1397] T 5474259a59a64b3ea875c6eca0e2eebb P 7e334a6d68eb4d98853ce401bd25949b: Time spent bootstrapping tablet: real 0.006s user 0.003s sys 0.000s
I20260806 04:57:25.000535 12936 tablet_bootstrap.cc:492] T 5474259a59a64b3ea875c6eca0e2eebb P 61f45f84da5f4e89bd2a27ed0cdc051b: No bootstrap required, opened a new log
I20260806 04:57:25.000665 12936 ts_tablet_manager.cc:1397] T 5474259a59a64b3ea875c6eca0e2eebb P 61f45f84da5f4e89bd2a27ed0cdc051b: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260806 04:57:25.001427 12936 raft_consensus.cc:348] T 5474259a59a64b3ea875c6eca0e2eebb P 61f45f84da5f4e89bd2a27ed0cdc051b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b26fa9b4a984025a159d0f38be81428" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 41697 } } peers { permanent_uuid: "61f45f84da5f4e89bd2a27ed0cdc051b" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 40541 } } peers { permanent_uuid: "7e334a6d68eb4d98853ce401bd25949b" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 34229 } }
I20260806 04:57:25.001397 12937 raft_consensus.cc:348] T 5474259a59a64b3ea875c6eca0e2eebb P 7e334a6d68eb4d98853ce401bd25949b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b26fa9b4a984025a159d0f38be81428" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 41697 } } peers { permanent_uuid: "61f45f84da5f4e89bd2a27ed0cdc051b" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 40541 } } peers { permanent_uuid: "7e334a6d68eb4d98853ce401bd25949b" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 34229 } }
I20260806 04:57:25.001648 12936 raft_consensus.cc:374] T 5474259a59a64b3ea875c6eca0e2eebb P 61f45f84da5f4e89bd2a27ed0cdc051b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 04:57:25.001724 12936 raft_consensus.cc:729] T 5474259a59a64b3ea875c6eca0e2eebb P 61f45f84da5f4e89bd2a27ed0cdc051b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 61f45f84da5f4e89bd2a27ed0cdc051b, State: Initialized, Role: FOLLOWER
I20260806 04:57:25.001720 12937 raft_consensus.cc:374] T 5474259a59a64b3ea875c6eca0e2eebb P 7e334a6d68eb4d98853ce401bd25949b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 04:57:25.001829 12937 raft_consensus.cc:729] T 5474259a59a64b3ea875c6eca0e2eebb P 7e334a6d68eb4d98853ce401bd25949b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e334a6d68eb4d98853ce401bd25949b, State: Initialized, Role: FOLLOWER
I20260806 04:57:25.001959 12936 consensus_queue.cc:260] T 5474259a59a64b3ea875c6eca0e2eebb P 61f45f84da5f4e89bd2a27ed0cdc051b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b26fa9b4a984025a159d0f38be81428" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 41697 } } peers { permanent_uuid: "61f45f84da5f4e89bd2a27ed0cdc051b" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 40541 } } peers { permanent_uuid: "7e334a6d68eb4d98853ce401bd25949b" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 34229 } }
I20260806 04:57:25.002082 12937 consensus_queue.cc:260] T 5474259a59a64b3ea875c6eca0e2eebb P 7e334a6d68eb4d98853ce401bd25949b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b26fa9b4a984025a159d0f38be81428" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 41697 } } peers { permanent_uuid: "61f45f84da5f4e89bd2a27ed0cdc051b" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 40541 } } peers { permanent_uuid: "7e334a6d68eb4d98853ce401bd25949b" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 34229 } }
I20260806 04:57:25.002907 12937 ts_tablet_manager.cc:1428] T 5474259a59a64b3ea875c6eca0e2eebb P 7e334a6d68eb4d98853ce401bd25949b: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260806 04:57:25.002658 12943 tablet_replica.cc:439] T 5474259a59a64b3ea875c6eca0e2eebb P 7e334a6d68eb4d98853ce401bd25949b: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b26fa9b4a984025a159d0f38be81428" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 41697 } } peers { permanent_uuid: "61f45f84da5f4e89bd2a27ed0cdc051b" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 40541 } } peers { permanent_uuid: "7e334a6d68eb4d98853ce401bd25949b" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 34229 } } }
I20260806 04:57:25.003054 12943 tablet_replica.cc:442] T 5474259a59a64b3ea875c6eca0e2eebb P 7e334a6d68eb4d98853ce401bd25949b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 04:57:25.003719 12944 tablet_replica.cc:439] T 5474259a59a64b3ea875c6eca0e2eebb P 61f45f84da5f4e89bd2a27ed0cdc051b: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b26fa9b4a984025a159d0f38be81428" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 41697 } } peers { permanent_uuid: "61f45f84da5f4e89bd2a27ed0cdc051b" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 40541 } } peers { permanent_uuid: "7e334a6d68eb4d98853ce401bd25949b" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 34229 } } }
I20260806 04:57:25.004105 12944 tablet_replica.cc:442] T 5474259a59a64b3ea875c6eca0e2eebb P 61f45f84da5f4e89bd2a27ed0cdc051b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 04:57:25.005515 12936 ts_tablet_manager.cc:1428] T 5474259a59a64b3ea875c6eca0e2eebb P 61f45f84da5f4e89bd2a27ed0cdc051b: Time spent starting tablet: real 0.005s user 0.003s sys 0.000s
I20260806 04:57:25.006699 12939 tablet_bootstrap.cc:492] T 5474259a59a64b3ea875c6eca0e2eebb P 4b26fa9b4a984025a159d0f38be81428: No bootstrap required, opened a new log
I20260806 04:57:25.006868 12939 ts_tablet_manager.cc:1397] T 5474259a59a64b3ea875c6eca0e2eebb P 4b26fa9b4a984025a159d0f38be81428: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260806 04:57:25.007648 12939 raft_consensus.cc:348] T 5474259a59a64b3ea875c6eca0e2eebb P 4b26fa9b4a984025a159d0f38be81428 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b26fa9b4a984025a159d0f38be81428" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 41697 } } peers { permanent_uuid: "61f45f84da5f4e89bd2a27ed0cdc051b" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 40541 } } peers { permanent_uuid: "7e334a6d68eb4d98853ce401bd25949b" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 34229 } }
I20260806 04:57:25.007907 12939 raft_consensus.cc:374] T 5474259a59a64b3ea875c6eca0e2eebb P 4b26fa9b4a984025a159d0f38be81428 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 04:57:25.007983 12939 raft_consensus.cc:729] T 5474259a59a64b3ea875c6eca0e2eebb P 4b26fa9b4a984025a159d0f38be81428 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b26fa9b4a984025a159d0f38be81428, State: Initialized, Role: FOLLOWER
I20260806 04:57:25.008231 12939 consensus_queue.cc:260] T 5474259a59a64b3ea875c6eca0e2eebb P 4b26fa9b4a984025a159d0f38be81428 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b26fa9b4a984025a159d0f38be81428" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 41697 } } peers { permanent_uuid: "61f45f84da5f4e89bd2a27ed0cdc051b" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 40541 } } peers { permanent_uuid: "7e334a6d68eb4d98853ce401bd25949b" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 34229 } }
I20260806 04:57:25.008873 12939 ts_tablet_manager.cc:1428] T 5474259a59a64b3ea875c6eca0e2eebb P 4b26fa9b4a984025a159d0f38be81428: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260806 04:57:25.008708 12945 tablet_replica.cc:439] T 5474259a59a64b3ea875c6eca0e2eebb P 4b26fa9b4a984025a159d0f38be81428: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b26fa9b4a984025a159d0f38be81428" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 41697 } } peers { permanent_uuid: "61f45f84da5f4e89bd2a27ed0cdc051b" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 40541 } } peers { permanent_uuid: "7e334a6d68eb4d98853ce401bd25949b" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 34229 } } }
I20260806 04:57:25.009191 12945 tablet_replica.cc:442] T 5474259a59a64b3ea875c6eca0e2eebb P 4b26fa9b4a984025a159d0f38be81428: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 04:57:25.208174 12945 raft_consensus.cc:482] T 5474259a59a64b3ea875c6eca0e2eebb P 4b26fa9b4a984025a159d0f38be81428 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 04:57:25.208441 12945 raft_consensus.cc:504] T 5474259a59a64b3ea875c6eca0e2eebb P 4b26fa9b4a984025a159d0f38be81428 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b26fa9b4a984025a159d0f38be81428" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 41697 } } peers { permanent_uuid: "61f45f84da5f4e89bd2a27ed0cdc051b" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 40541 } } peers { permanent_uuid: "7e334a6d68eb4d98853ce401bd25949b" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 34229 } }
I20260806 04:57:25.209975 12945 leader_election.cc:288] T 5474259a59a64b3ea875c6eca0e2eebb P 4b26fa9b4a984025a159d0f38be81428 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 61f45f84da5f4e89bd2a27ed0cdc051b (127.9.96.66:40541), 7e334a6d68eb4d98853ce401bd25949b (127.9.96.67:34229)
I20260806 04:57:25.219604 12931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:57:25.210516 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.96.65:46651 (local address 127.9.96.67:34229)
0806 04:57:25.210728 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0806 04:57:25.210734 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 04:57:25.211445 (+ 711us) server_negotiation.cc:407] Connection header received
0806 04:57:25.211696 (+ 251us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:57:25.211701 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:57:25.211786 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:57:25.211896 (+ 110us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 04:57:25.212906 (+ 1010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:57:25.214045 (+ 1139us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:57:25.217183 (+ 3138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:57:25.218414 (+ 1231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:57:25.218556 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:57:25.218658 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0806 04:57:25.219179 (+ 521us) server_negotiation.cc:299] Negotiation successful
0806 04:57:25.219318 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260806 04:57:25.219604 12929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:57:25.210621 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.96.65:44529 (local address 127.9.96.66:40541)
0806 04:57:25.210845 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0806 04:57:25.210851 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 04:57:25.211484 (+ 633us) server_negotiation.cc:407] Connection header received
0806 04:57:25.211711 (+ 227us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:57:25.211716 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:57:25.211786 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:57:25.211905 (+ 119us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 04:57:25.212908 (+ 1003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:57:25.214027 (+ 1119us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:57:25.217189 (+ 3162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:57:25.218414 (+ 1225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:57:25.218556 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:57:25.218665 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0806 04:57:25.219139 (+ 474us) server_negotiation.cc:299] Negotiation successful
0806 04:57:25.219328 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260806 04:57:25.219754 12963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:57:25.210272 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.67:34229 (local address 127.9.96.65:46651)
0806 04:57:25.211273 (+ 1001us) negotiation.cc:107] Waiting for socket to connect
0806 04:57:25.211294 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0806 04:57:25.211524 (+ 230us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:57:25.211961 (+ 437us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:57:25.211966 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:57:25.212000 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 04:57:25.212692 (+ 692us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:57:25.212707 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:57:25.214192 (+ 1485us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:57:25.214197 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:57:25.217012 (+ 2815us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:57:25.217028 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:57:25.218705 (+ 1677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:57:25.218710 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:57:25.218922 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:57:25.218978 (+ 56us) client_negotiation.cc:769] Sending connection context
0806 04:57:25.219131 (+ 153us) client_negotiation.cc:240] Negotiation successful
0806 04:57:25.219316 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":844,"mutex_wait_us":294,"thread_start_us":754,"threads_started":1}
I20260806 04:57:25.219938 12962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:57:25.210293 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.66:40541 (local address 127.9.96.65:44529)
0806 04:57:25.211358 (+ 1065us) negotiation.cc:107] Waiting for socket to connect
0806 04:57:25.211399 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0806 04:57:25.211568 (+ 169us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:57:25.211961 (+ 393us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:57:25.211966 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:57:25.212066 (+ 100us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 04:57:25.212701 (+ 635us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:57:25.212710 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:57:25.214192 (+ 1482us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:57:25.214197 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:57:25.217012 (+ 2815us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:57:25.217028 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:57:25.218690 (+ 1662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:57:25.218697 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:57:25.218928 (+ 231us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:57:25.218980 (+ 52us) client_negotiation.cc:769] Sending connection context
0806 04:57:25.219081 (+ 101us) client_negotiation.cc:240] Negotiation successful
0806 04:57:25.219243 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":938,"mutex_wait_us":272,"spinlock_wait_cycles":24320,"thread_start_us":635,"threads_started":1}
I20260806 04:57:25.220476 12845 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5474259a59a64b3ea875c6eca0e2eebb" candidate_uuid: "4b26fa9b4a984025a159d0f38be81428" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e334a6d68eb4d98853ce401bd25949b" is_pre_election: true
I20260806 04:57:25.220781 12845 raft_consensus.cc:2454] T 5474259a59a64b3ea875c6eca0e2eebb P 7e334a6d68eb4d98853ce401bd25949b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b26fa9b4a984025a159d0f38be81428 in term 0.
I20260806 04:57:25.221304 12339 leader_election.cc:302] T 5474259a59a64b3ea875c6eca0e2eebb P 4b26fa9b4a984025a159d0f38be81428 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4b26fa9b4a984025a159d0f38be81428, 7e334a6d68eb4d98853ce401bd25949b; no voters:
I20260806 04:57:25.221448 12640 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5474259a59a64b3ea875c6eca0e2eebb" candidate_uuid: "4b26fa9b4a984025a159d0f38be81428" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "61f45f84da5f4e89bd2a27ed0cdc051b" is_pre_election: true
I20260806 04:57:25.221776 12945 raft_consensus.cc:2781] T 5474259a59a64b3ea875c6eca0e2eebb P 4b26fa9b4a984025a159d0f38be81428 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 04:57:25.221927 12945 raft_consensus.cc:482] T 5474259a59a64b3ea875c6eca0e2eebb P 4b26fa9b4a984025a159d0f38be81428 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 04:57:25.221901 12640 raft_consensus.cc:2454] T 5474259a59a64b3ea875c6eca0e2eebb P 61f45f84da5f4e89bd2a27ed0cdc051b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b26fa9b4a984025a159d0f38be81428 in term 0.
I20260806 04:57:25.222046 12945 raft_consensus.cc:3037] T 5474259a59a64b3ea875c6eca0e2eebb P 4b26fa9b4a984025a159d0f38be81428 [term 0 FOLLOWER]: Advancing to term 1
I20260806 04:57:25.224650 12945 raft_consensus.cc:504] T 5474259a59a64b3ea875c6eca0e2eebb P 4b26fa9b4a984025a159d0f38be81428 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b26fa9b4a984025a159d0f38be81428" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 41697 } } peers { permanent_uuid: "61f45f84da5f4e89bd2a27ed0cdc051b" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 40541 } } peers { permanent_uuid: "7e334a6d68eb4d98853ce401bd25949b" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 34229 } }
I20260806 04:57:25.225250 12945 leader_election.cc:288] T 5474259a59a64b3ea875c6eca0e2eebb P 4b26fa9b4a984025a159d0f38be81428 [CANDIDATE]: Term 1 election: Requested vote from peers 61f45f84da5f4e89bd2a27ed0cdc051b (127.9.96.66:40541), 7e334a6d68eb4d98853ce401bd25949b (127.9.96.67:34229)
I20260806 04:57:25.225713 12640 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5474259a59a64b3ea875c6eca0e2eebb" candidate_uuid: "4b26fa9b4a984025a159d0f38be81428" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "61f45f84da5f4e89bd2a27ed0cdc051b"
I20260806 04:57:25.225723 12845 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5474259a59a64b3ea875c6eca0e2eebb" candidate_uuid: "4b26fa9b4a984025a159d0f38be81428" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e334a6d68eb4d98853ce401bd25949b"
I20260806 04:57:25.225894 12640 raft_consensus.cc:3037] T 5474259a59a64b3ea875c6eca0e2eebb P 61f45f84da5f4e89bd2a27ed0cdc051b [term 0 FOLLOWER]: Advancing to term 1
I20260806 04:57:25.225937 12845 raft_consensus.cc:3037] T 5474259a59a64b3ea875c6eca0e2eebb P 7e334a6d68eb4d98853ce401bd25949b [term 0 FOLLOWER]: Advancing to term 1
I20260806 04:57:25.228012 12845 raft_consensus.cc:2454] T 5474259a59a64b3ea875c6eca0e2eebb P 7e334a6d68eb4d98853ce401bd25949b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b26fa9b4a984025a159d0f38be81428 in term 1.
I20260806 04:57:25.228123 12640 raft_consensus.cc:2454] T 5474259a59a64b3ea875c6eca0e2eebb P 61f45f84da5f4e89bd2a27ed0cdc051b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b26fa9b4a984025a159d0f38be81428 in term 1.
I20260806 04:57:25.228524 12339 leader_election.cc:302] T 5474259a59a64b3ea875c6eca0e2eebb P 4b26fa9b4a984025a159d0f38be81428 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4b26fa9b4a984025a159d0f38be81428, 7e334a6d68eb4d98853ce401bd25949b; no voters:
I20260806 04:57:25.228856 12945 raft_consensus.cc:2781] T 5474259a59a64b3ea875c6eca0e2eebb P 4b26fa9b4a984025a159d0f38be81428 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 04:57:25.229326 12945 raft_consensus.cc:686] T 5474259a59a64b3ea875c6eca0e2eebb P 4b26fa9b4a984025a159d0f38be81428 [term 1 LEADER]: Becoming Leader. State: Replica: 4b26fa9b4a984025a159d0f38be81428, State: Running, Role: LEADER
I20260806 04:57:25.229787 12945 consensus_queue.cc:237] T 5474259a59a64b3ea875c6eca0e2eebb P 4b26fa9b4a984025a159d0f38be81428 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b26fa9b4a984025a159d0f38be81428" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 41697 } } peers { permanent_uuid: "61f45f84da5f4e89bd2a27ed0cdc051b" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 40541 } } peers { permanent_uuid: "7e334a6d68eb4d98853ce401bd25949b" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 34229 } }
I20260806 04:57:25.233356 12964 tablet_replica.cc:439] T 5474259a59a64b3ea875c6eca0e2eebb P 4b26fa9b4a984025a159d0f38be81428: TxnStatusTablet state changed. Reason: New leader 4b26fa9b4a984025a159d0f38be81428. Latest consensus state: current_term: 1 leader_uuid: "4b26fa9b4a984025a159d0f38be81428" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b26fa9b4a984025a159d0f38be81428" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 41697 } } peers { permanent_uuid: "61f45f84da5f4e89bd2a27ed0cdc051b" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 40541 } } peers { permanent_uuid: "7e334a6d68eb4d98853ce401bd25949b" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 34229 } } }
I20260806 04:57:25.233793 12964 tablet_replica.cc:442] T 5474259a59a64b3ea875c6eca0e2eebb P 4b26fa9b4a984025a159d0f38be81428: This TxnStatusTablet replica's current role is: LEADER
I20260806 04:57:25.234509 12966 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260806 04:57:25.234812 12966 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260806 04:57:25.234778 12210 catalog_manager.cc:5495] T 5474259a59a64b3ea875c6eca0e2eebb P 4b26fa9b4a984025a159d0f38be81428 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4b26fa9b4a984025a159d0f38be81428 (127.9.96.65). New cstate: current_term: 1 leader_uuid: "4b26fa9b4a984025a159d0f38be81428" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b26fa9b4a984025a159d0f38be81428" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 41697 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "61f45f84da5f4e89bd2a27ed0cdc051b" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 40541 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7e334a6d68eb4d98853ce401bd25949b" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 34229 } health_report { overall_health: UNKNOWN } } }
I20260806 04:57:25.302945 12955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:57:25.296451 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.65:41697 (local address 127.0.0.1:50028)
0806 04:57:25.296809 (+ 358us) negotiation.cc:107] Waiting for socket to connect
0806 04:57:25.296827 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 04:57:25.296936 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:57:25.297504 (+ 568us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:57:25.297511 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:57:25.297525 (+ 14us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 04:57:25.297764 (+ 239us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:57:25.297776 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:57:25.299244 (+ 1468us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:57:25.299248 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:57:25.300299 (+ 1051us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:57:25.300311 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:57:25.301263 (+ 952us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:57:25.301268 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:57:25.301422 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:57:25.301474 (+ 52us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 04:57:25.302505 (+ 1031us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 04:57:25.302520 (+ 15us) client_negotiation.cc:769] Sending connection context
0806 04:57:25.302568 (+ 48us) client_negotiation.cc:240] Negotiation successful
0806 04:57:25.302685 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130}
I20260806 04:57:25.303053 12934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:57:25.296660 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50028 (local address 127.9.96.65:41697)
0806 04:57:25.296862 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0806 04:57:25.296868 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 04:57:25.296889 (+ 21us) server_negotiation.cc:407] Connection header received
0806 04:57:25.297141 (+ 252us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:57:25.297147 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:57:25.297300 (+ 153us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:57:25.297472 (+ 172us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 04:57:25.297906 (+ 434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:57:25.299111 (+ 1205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:57:25.300442 (+ 1331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:57:25.301092 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:57:25.301226 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:57:25.301662 (+ 436us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 04:57:25.302392 (+ 730us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 04:57:25.302543 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0806 04:57:25.302611 (+ 68us) server_negotiation.cc:299] Negotiation successful
0806 04:57:25.302702 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260806 04:57:25.310364 12640 raft_consensus.cc:1261] T 5474259a59a64b3ea875c6eca0e2eebb P 61f45f84da5f4e89bd2a27ed0cdc051b [term 1 FOLLOWER]: Refusing update from remote peer 4b26fa9b4a984025a159d0f38be81428: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 04:57:25.310755 12845 raft_consensus.cc:1261] T 5474259a59a64b3ea875c6eca0e2eebb P 7e334a6d68eb4d98853ce401bd25949b [term 1 FOLLOWER]: Refusing update from remote peer 4b26fa9b4a984025a159d0f38be81428: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 04:57:25.311432 12964 consensus_queue.cc:1035] T 5474259a59a64b3ea875c6eca0e2eebb P 4b26fa9b4a984025a159d0f38be81428 [LEADER]: Connected to new peer: Peer: permanent_uuid: "61f45f84da5f4e89bd2a27ed0cdc051b" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 40541 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 04:57:25.311898 12945 consensus_queue.cc:1035] T 5474259a59a64b3ea875c6eca0e2eebb P 4b26fa9b4a984025a159d0f38be81428 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e334a6d68eb4d98853ce401bd25949b" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 34229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 04:57:25.327394 12943 tablet_replica.cc:439] T 5474259a59a64b3ea875c6eca0e2eebb P 7e334a6d68eb4d98853ce401bd25949b: TxnStatusTablet state changed. Reason: New leader 4b26fa9b4a984025a159d0f38be81428. Latest consensus state: current_term: 1 leader_uuid: "4b26fa9b4a984025a159d0f38be81428" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b26fa9b4a984025a159d0f38be81428" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 41697 } } peers { permanent_uuid: "61f45f84da5f4e89bd2a27ed0cdc051b" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 40541 } } peers { permanent_uuid: "7e334a6d68eb4d98853ce401bd25949b" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 34229 } } }
I20260806 04:57:25.327708 12943 tablet_replica.cc:442] T 5474259a59a64b3ea875c6eca0e2eebb P 7e334a6d68eb4d98853ce401bd25949b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 04:57:25.330967 12944 tablet_replica.cc:439] T 5474259a59a64b3ea875c6eca0e2eebb P 61f45f84da5f4e89bd2a27ed0cdc051b: TxnStatusTablet state changed. Reason: New leader 4b26fa9b4a984025a159d0f38be81428. Latest consensus state: current_term: 1 leader_uuid: "4b26fa9b4a984025a159d0f38be81428" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b26fa9b4a984025a159d0f38be81428" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 41697 } } peers { permanent_uuid: "61f45f84da5f4e89bd2a27ed0cdc051b" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 40541 } } peers { permanent_uuid: "7e334a6d68eb4d98853ce401bd25949b" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 34229 } } }
I20260806 04:57:25.331357 12944 tablet_replica.cc:442] T 5474259a59a64b3ea875c6eca0e2eebb P 61f45f84da5f4e89bd2a27ed0cdc051b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 04:57:25.335767 12974 mvcc.cc:205] Tried to move back new op lower bound from 7315424236781510656 to 7315424236465225728. Current Snapshot: MvccSnapshot[applied={T|T < 7315424236781510656}]
I20260806 04:57:25.341346 12944 tablet_replica.cc:439] T 5474259a59a64b3ea875c6eca0e2eebb P 61f45f84da5f4e89bd2a27ed0cdc051b: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "4b26fa9b4a984025a159d0f38be81428" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b26fa9b4a984025a159d0f38be81428" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 41697 } } peers { permanent_uuid: "61f45f84da5f4e89bd2a27ed0cdc051b" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 40541 } } peers { permanent_uuid: "7e334a6d68eb4d98853ce401bd25949b" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 34229 } } }
I20260806 04:57:25.341629 12944 tablet_replica.cc:442] T 5474259a59a64b3ea875c6eca0e2eebb P 61f45f84da5f4e89bd2a27ed0cdc051b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 04:57:25.343062 12943 tablet_replica.cc:439] T 5474259a59a64b3ea875c6eca0e2eebb P 7e334a6d68eb4d98853ce401bd25949b: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "4b26fa9b4a984025a159d0f38be81428" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b26fa9b4a984025a159d0f38be81428" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 41697 } } peers { permanent_uuid: "61f45f84da5f4e89bd2a27ed0cdc051b" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 40541 } } peers { permanent_uuid: "7e334a6d68eb4d98853ce401bd25949b" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 34229 } } }
I20260806 04:57:25.343406 12943 tablet_replica.cc:442] T 5474259a59a64b3ea875c6eca0e2eebb P 7e334a6d68eb4d98853ce401bd25949b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 04:57:25.345753 12945 tablet_replica.cc:439] T 5474259a59a64b3ea875c6eca0e2eebb P 4b26fa9b4a984025a159d0f38be81428: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "4b26fa9b4a984025a159d0f38be81428" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b26fa9b4a984025a159d0f38be81428" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 41697 } } peers { permanent_uuid: "61f45f84da5f4e89bd2a27ed0cdc051b" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 40541 } } peers { permanent_uuid: "7e334a6d68eb4d98853ce401bd25949b" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 34229 } } }
I20260806 04:57:25.346053 12945 tablet_replica.cc:442] T 5474259a59a64b3ea875c6eca0e2eebb P 4b26fa9b4a984025a159d0f38be81428: This TxnStatusTablet replica's current role is: LEADER
I20260806 04:57:25.350804 12982 tablet_replica.cc:439] T 5474259a59a64b3ea875c6eca0e2eebb P 4b26fa9b4a984025a159d0f38be81428: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "4b26fa9b4a984025a159d0f38be81428" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b26fa9b4a984025a159d0f38be81428" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 41697 } } peers { permanent_uuid: "61f45f84da5f4e89bd2a27ed0cdc051b" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 40541 } } peers { permanent_uuid: "7e334a6d68eb4d98853ce401bd25949b" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 34229 } } }
I20260806 04:57:25.353012 12982 tablet_replica.cc:442] T 5474259a59a64b3ea875c6eca0e2eebb P 4b26fa9b4a984025a159d0f38be81428: This TxnStatusTablet replica's current role is: LEADER
I20260806 04:57:25.361119 12929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:57:25.354532 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44734 (local address 127.9.96.66:40541)
0806 04:57:25.354726 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0806 04:57:25.354732 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 04:57:25.355430 (+ 698us) server_negotiation.cc:407] Connection header received
0806 04:57:25.355693 (+ 263us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:57:25.355701 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:57:25.355809 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:57:25.355975 (+ 166us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 04:57:25.356482 (+ 507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:57:25.357325 (+ 843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:57:25.358835 (+ 1510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:57:25.359295 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:57:25.359494 (+ 199us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:57:25.359933 (+ 439us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 04:57:25.360459 (+ 526us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 04:57:25.360630 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0806 04:57:25.360722 (+ 92us) server_negotiation.cc:299] Negotiation successful
0806 04:57:25.360807 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260806 04:57:25.378515 12934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:57:25.372158 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.96.66:57785 (local address 127.9.96.65:41697)
0806 04:57:25.372340 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0806 04:57:25.372346 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 04:57:25.372751 (+ 405us) server_negotiation.cc:407] Connection header received
0806 04:57:25.372947 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:57:25.372951 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:57:25.373017 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:57:25.373096 (+ 79us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 04:57:25.373678 (+ 582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:57:25.374441 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:57:25.376628 (+ 2187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:57:25.377120 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:57:25.377231 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:57:25.377840 (+ 609us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 04:57:25.377949 (+ 109us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 04:57:25.378034 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0806 04:57:25.378230 (+ 196us) server_negotiation.cc:299] Negotiation successful
0806 04:57:25.378296 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260806 04:57:25.378705 12989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:57:25.371859 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.65:41697 (local address 127.9.96.66:57785)
0806 04:57:25.372614 (+ 755us) negotiation.cc:107] Waiting for socket to connect
0806 04:57:25.372632 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 04:57:25.372778 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:57:25.373143 (+ 365us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:57:25.373148 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:57:25.373162 (+ 14us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 04:57:25.373521 (+ 359us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:57:25.373533 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:57:25.374578 (+ 1045us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:57:25.374584 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:57:25.376459 (+ 1875us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:57:25.376475 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:57:25.377309 (+ 834us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:57:25.377317 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:57:25.377497 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:57:25.377563 (+ 66us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 04:57:25.378062 (+ 499us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 04:57:25.378073 (+ 11us) client_negotiation.cc:769] Sending connection context
0806 04:57:25.378156 (+ 83us) client_negotiation.cc:240] Negotiation successful
0806 04:57:25.378315 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":587,"thread_start_us":291,"threads_started":1}
I20260806 04:57:25.388150 12845 raft_consensus.cc:1261] T f3288924f5d24b46b6924e00565f996a P 7e334a6d68eb4d98853ce401bd25949b [term 1 FOLLOWER]: Refusing update from remote peer 61f45f84da5f4e89bd2a27ed0cdc051b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 04:57:25.388495 12441 raft_consensus.cc:1261] T f3288924f5d24b46b6924e00565f996a P 4b26fa9b4a984025a159d0f38be81428 [term 1 FOLLOWER]: Refusing update from remote peer 61f45f84da5f4e89bd2a27ed0cdc051b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 04:57:25.388902 12948 consensus_queue.cc:1035] T f3288924f5d24b46b6924e00565f996a P 61f45f84da5f4e89bd2a27ed0cdc051b [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e334a6d68eb4d98853ce401bd25949b" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 34229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 04:57:25.389346 12944 consensus_queue.cc:1035] T f3288924f5d24b46b6924e00565f996a P 61f45f84da5f4e89bd2a27ed0cdc051b [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b26fa9b4a984025a159d0f38be81428" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 41697 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 04:57:25.427018 12929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:57:25.420457 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.96.65:58539 (local address 127.9.96.66:40541)
0806 04:57:25.420656 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0806 04:57:25.420662 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 04:57:25.421260 (+ 598us) server_negotiation.cc:407] Connection header received
0806 04:57:25.421459 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:57:25.421466 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:57:25.421558 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:57:25.421648 (+ 90us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 04:57:25.422250 (+ 602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:57:25.423401 (+ 1151us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:57:25.425125 (+ 1724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:57:25.425710 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:57:25.425844 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:57:25.426312 (+ 468us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 04:57:25.426410 (+ 98us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 04:57:25.426510 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0806 04:57:25.426641 (+ 131us) server_negotiation.cc:299] Negotiation successful
0806 04:57:25.426718 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260806 04:57:25.427140 12997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:57:25.420021 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.66:40541 (local address 127.9.96.65:58539)
0806 04:57:25.421109 (+ 1088us) negotiation.cc:107] Waiting for socket to connect
0806 04:57:25.421133 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0806 04:57:25.421319 (+ 186us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:57:25.421687 (+ 368us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:57:25.421692 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:57:25.421709 (+ 17us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 04:57:25.422077 (+ 368us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:57:25.422087 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:57:25.423564 (+ 1477us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:57:25.423569 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:57:25.424966 (+ 1397us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:57:25.424980 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:57:25.425908 (+ 928us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:57:25.425917 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:57:25.426074 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:57:25.426126 (+ 52us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 04:57:25.426513 (+ 387us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 04:57:25.426527 (+ 14us) client_negotiation.cc:769] Sending connection context
0806 04:57:25.426610 (+ 83us) client_negotiation.cc:240] Negotiation successful
0806 04:57:25.426743 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":903,"thread_start_us":665,"threads_started":1}
I20260806 04:57:25.464809 12210 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47864:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 04:57:25.465085 12210 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47864:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 04:57:25.469033 12210 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ebfef4bb062640d9bf626f0e62f6b98c: Sending DeleteTablet for 3 replicas of tablet f3288924f5d24b46b6924e00565f996a
I20260806 04:57:25.470332 12609 tablet_service.cc:1514] Processing DeleteTablet for tablet f3288924f5d24b46b6924e00565f996a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 04:57:25 UTC) from {username='kudu', principal='kudu/127.9.96.126@KRBTEST.COM'} at 127.0.0.1:44726
I20260806 04:57:25.471386 12408 tablet_service.cc:1514] Processing DeleteTablet for tablet f3288924f5d24b46b6924e00565f996a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 04:57:25 UTC) from {username='kudu', principal='kudu/127.9.96.126@KRBTEST.COM'} at 127.0.0.1:50012
I20260806 04:57:25.471421 12808 tablet_service.cc:1514] Processing DeleteTablet for tablet f3288924f5d24b46b6924e00565f996a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 04:57:25 UTC) from {username='kudu', principal='kudu/127.9.96.126@KRBTEST.COM'} at 127.0.0.1:60914
I20260806 04:57:25.472282 12210 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47864:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 04:57:25.472862 12210 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 04:57:25.479969 13005 tablet_replica.cc:330] T f3288924f5d24b46b6924e00565f996a P 4b26fa9b4a984025a159d0f38be81428: stopping tablet replica
I20260806 04:57:25.480512 13005 raft_consensus.cc:2229] T f3288924f5d24b46b6924e00565f996a P 4b26fa9b4a984025a159d0f38be81428 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 04:57:25.480916 13005 raft_consensus.cc:2258] T f3288924f5d24b46b6924e00565f996a P 4b26fa9b4a984025a159d0f38be81428 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 04:57:25.481386 12608 tablet_service.cc:1467] Processing CreateTablet for tablet 478014fd7c524f18b0f86961e6143256 (DEFAULT_TABLE table=test-table [id=fec0c04e05b74f78a6328c5bc1eed5c0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 04:57:25.481688 13004 tablet_replica.cc:330] T f3288924f5d24b46b6924e00565f996a P 7e334a6d68eb4d98853ce401bd25949b: stopping tablet replica
I20260806 04:57:25.481989 12608 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 478014fd7c524f18b0f86961e6143256. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 04:57:25.482134 13004 raft_consensus.cc:2229] T f3288924f5d24b46b6924e00565f996a P 7e334a6d68eb4d98853ce401bd25949b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 04:57:25.483763 12806 tablet_service.cc:1467] Processing CreateTablet for tablet 478014fd7c524f18b0f86961e6143256 (DEFAULT_TABLE table=test-table [id=fec0c04e05b74f78a6328c5bc1eed5c0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 04:57:25.484159 13004 raft_consensus.cc:2258] T f3288924f5d24b46b6924e00565f996a P 7e334a6d68eb4d98853ce401bd25949b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 04:57:25.484365 12806 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 478014fd7c524f18b0f86961e6143256. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 04:57:25.486126 13003 tablet_replica.cc:330] T f3288924f5d24b46b6924e00565f996a P 61f45f84da5f4e89bd2a27ed0cdc051b: stopping tablet replica
I20260806 04:57:25.486610 13003 raft_consensus.cc:2229] T f3288924f5d24b46b6924e00565f996a P 61f45f84da5f4e89bd2a27ed0cdc051b [term 1 LEADER]: Raft consensus shutting down.
I20260806 04:57:25.487310 13003 raft_consensus.cc:2258] T f3288924f5d24b46b6924e00565f996a P 61f45f84da5f4e89bd2a27ed0cdc051b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 04:57:25.487875 13005 ts_tablet_manager.cc:1905] T f3288924f5d24b46b6924e00565f996a P 4b26fa9b4a984025a159d0f38be81428: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 04:57:25.489182 13003 ts_tablet_manager.cc:1905] T f3288924f5d24b46b6924e00565f996a P 61f45f84da5f4e89bd2a27ed0cdc051b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 04:57:25.495903 12937 tablet_bootstrap.cc:492] T 478014fd7c524f18b0f86961e6143256 P 7e334a6d68eb4d98853ce401bd25949b: Bootstrap starting.
I20260806 04:57:25.496893 13005 ts_tablet_manager.cc:1918] T f3288924f5d24b46b6924e00565f996a P 4b26fa9b4a984025a159d0f38be81428: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 04:57:25.497129 13005 log.cc:1196] T f3288924f5d24b46b6924e00565f996a P 4b26fa9b4a984025a159d0f38be81428: Deleting WAL directory at /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785992227050689-9601-0/minicluster-data/ts-0/wal/wals/f3288924f5d24b46b6924e00565f996a
I20260806 04:57:25.497113 13004 ts_tablet_manager.cc:1905] T f3288924f5d24b46b6924e00565f996a P 7e334a6d68eb4d98853ce401bd25949b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 04:57:25.497524 12937 tablet_bootstrap.cc:654] T 478014fd7c524f18b0f86961e6143256 P 7e334a6d68eb4d98853ce401bd25949b: Neither blocks nor log segments found. Creating new log.
I20260806 04:57:25.497892 13005 ts_tablet_manager.cc:1939] T f3288924f5d24b46b6924e00565f996a P 4b26fa9b4a984025a159d0f38be81428: Deleting consensus metadata
I20260806 04:57:25.499811 12187 catalog_manager.cc:4841] TS 4b26fa9b4a984025a159d0f38be81428 (127.9.96.65:41697): tablet f3288924f5d24b46b6924e00565f996a (table test-table [id=6a3703d849064091a2fdaf55a08cb080]) successfully deleted
I20260806 04:57:25.500408 13003 ts_tablet_manager.cc:1918] T f3288924f5d24b46b6924e00565f996a P 61f45f84da5f4e89bd2a27ed0cdc051b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 04:57:25.500561 13003 log.cc:1196] T f3288924f5d24b46b6924e00565f996a P 61f45f84da5f4e89bd2a27ed0cdc051b: Deleting WAL directory at /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785992227050689-9601-0/minicluster-data/ts-1/wal/wals/f3288924f5d24b46b6924e00565f996a
I20260806 04:57:25.501082 13003 ts_tablet_manager.cc:1939] T f3288924f5d24b46b6924e00565f996a P 61f45f84da5f4e89bd2a27ed0cdc051b: Deleting consensus metadata
I20260806 04:57:25.502696 12185 catalog_manager.cc:4841] TS 61f45f84da5f4e89bd2a27ed0cdc051b (127.9.96.66:40541): tablet f3288924f5d24b46b6924e00565f996a (table test-table [id=6a3703d849064091a2fdaf55a08cb080]) successfully deleted
I20260806 04:57:25.504627 13004 ts_tablet_manager.cc:1918] T f3288924f5d24b46b6924e00565f996a P 7e334a6d68eb4d98853ce401bd25949b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 04:57:25.504788 13004 log.cc:1196] T f3288924f5d24b46b6924e00565f996a P 7e334a6d68eb4d98853ce401bd25949b: Deleting WAL directory at /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785992227050689-9601-0/minicluster-data/ts-2/wal/wals/f3288924f5d24b46b6924e00565f996a
I20260806 04:57:25.505300 13004 ts_tablet_manager.cc:1939] T f3288924f5d24b46b6924e00565f996a P 7e334a6d68eb4d98853ce401bd25949b: Deleting consensus metadata
I20260806 04:57:25.507722 12187 catalog_manager.cc:4841] TS 7e334a6d68eb4d98853ce401bd25949b (127.9.96.67:34229): tablet f3288924f5d24b46b6924e00565f996a (table test-table [id=6a3703d849064091a2fdaf55a08cb080]) successfully deleted
I20260806 04:57:25.511677 12408 tablet_service.cc:1467] Processing CreateTablet for tablet 478014fd7c524f18b0f86961e6143256 (DEFAULT_TABLE table=test-table [id=fec0c04e05b74f78a6328c5bc1eed5c0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 04:57:25.512152 12408 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 478014fd7c524f18b0f86961e6143256. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 04:57:25.515028 13009 tablet_bootstrap.cc:492] T 478014fd7c524f18b0f86961e6143256 P 61f45f84da5f4e89bd2a27ed0cdc051b: Bootstrap starting.
I20260806 04:57:25.521238 13009 tablet_bootstrap.cc:654] T 478014fd7c524f18b0f86961e6143256 P 61f45f84da5f4e89bd2a27ed0cdc051b: Neither blocks nor log segments found. Creating new log.
I20260806 04:57:25.524151 12937 tablet_bootstrap.cc:492] T 478014fd7c524f18b0f86961e6143256 P 7e334a6d68eb4d98853ce401bd25949b: No bootstrap required, opened a new log
I20260806 04:57:25.524333 12937 ts_tablet_manager.cc:1397] T 478014fd7c524f18b0f86961e6143256 P 7e334a6d68eb4d98853ce401bd25949b: Time spent bootstrapping tablet: real 0.028s user 0.002s sys 0.007s
I20260806 04:57:25.525297 12937 raft_consensus.cc:348] T 478014fd7c524f18b0f86961e6143256 P 7e334a6d68eb4d98853ce401bd25949b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b26fa9b4a984025a159d0f38be81428" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 41697 } } peers { permanent_uuid: "61f45f84da5f4e89bd2a27ed0cdc051b" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 40541 } } peers { permanent_uuid: "7e334a6d68eb4d98853ce401bd25949b" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 34229 } }
I20260806 04:57:25.525568 12937 raft_consensus.cc:374] T 478014fd7c524f18b0f86961e6143256 P 7e334a6d68eb4d98853ce401bd25949b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 04:57:25.525642 12937 raft_consensus.cc:729] T 478014fd7c524f18b0f86961e6143256 P 7e334a6d68eb4d98853ce401bd25949b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e334a6d68eb4d98853ce401bd25949b, State: Initialized, Role: FOLLOWER
I20260806 04:57:25.525934 12937 consensus_queue.cc:260] T 478014fd7c524f18b0f86961e6143256 P 7e334a6d68eb4d98853ce401bd25949b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b26fa9b4a984025a159d0f38be81428" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 41697 } } peers { permanent_uuid: "61f45f84da5f4e89bd2a27ed0cdc051b" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 40541 } } peers { permanent_uuid: "7e334a6d68eb4d98853ce401bd25949b" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 34229 } }
I20260806 04:57:25.528898 12937 ts_tablet_manager.cc:1428] T 478014fd7c524f18b0f86961e6143256 P 7e334a6d68eb4d98853ce401bd25949b: Time spent starting tablet: real 0.004s user 0.001s sys 0.002s
I20260806 04:57:25.533241 13009 tablet_bootstrap.cc:492] T 478014fd7c524f18b0f86961e6143256 P 61f45f84da5f4e89bd2a27ed0cdc051b: No bootstrap required, opened a new log
I20260806 04:57:25.533443 13009 ts_tablet_manager.cc:1397] T 478014fd7c524f18b0f86961e6143256 P 61f45f84da5f4e89bd2a27ed0cdc051b: Time spent bootstrapping tablet: real 0.019s user 0.005s sys 0.005s
I20260806 04:57:25.534271 13009 raft_consensus.cc:348] T 478014fd7c524f18b0f86961e6143256 P 61f45f84da5f4e89bd2a27ed0cdc051b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b26fa9b4a984025a159d0f38be81428" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 41697 } } peers { permanent_uuid: "61f45f84da5f4e89bd2a27ed0cdc051b" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 40541 } } peers { permanent_uuid: "7e334a6d68eb4d98853ce401bd25949b" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 34229 } }
I20260806 04:57:25.534590 13009 raft_consensus.cc:374] T 478014fd7c524f18b0f86961e6143256 P 61f45f84da5f4e89bd2a27ed0cdc051b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 04:57:25.534723 13009 raft_consensus.cc:729] T 478014fd7c524f18b0f86961e6143256 P 61f45f84da5f4e89bd2a27ed0cdc051b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 61f45f84da5f4e89bd2a27ed0cdc051b, State: Initialized, Role: FOLLOWER
I20260806 04:57:25.535040 13009 consensus_queue.cc:260] T 478014fd7c524f18b0f86961e6143256 P 61f45f84da5f4e89bd2a27ed0cdc051b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b26fa9b4a984025a159d0f38be81428" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 41697 } } peers { permanent_uuid: "61f45f84da5f4e89bd2a27ed0cdc051b" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 40541 } } peers { permanent_uuid: "7e334a6d68eb4d98853ce401bd25949b" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 34229 } }
I20260806 04:57:25.537683 13014 tablet_bootstrap.cc:492] T 478014fd7c524f18b0f86961e6143256 P 4b26fa9b4a984025a159d0f38be81428: Bootstrap starting.
I20260806 04:57:25.539351 13009 ts_tablet_manager.cc:1428] T 478014fd7c524f18b0f86961e6143256 P 61f45f84da5f4e89bd2a27ed0cdc051b: Time spent starting tablet: real 0.006s user 0.001s sys 0.001s
I20260806 04:57:25.539764 13014 tablet_bootstrap.cc:654] T 478014fd7c524f18b0f86961e6143256 P 4b26fa9b4a984025a159d0f38be81428: Neither blocks nor log segments found. Creating new log.
I20260806 04:57:25.542829 13014 tablet_bootstrap.cc:492] T 478014fd7c524f18b0f86961e6143256 P 4b26fa9b4a984025a159d0f38be81428: No bootstrap required, opened a new log
I20260806 04:57:25.543040 13014 ts_tablet_manager.cc:1397] T 478014fd7c524f18b0f86961e6143256 P 4b26fa9b4a984025a159d0f38be81428: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260806 04:57:25.543915 13014 raft_consensus.cc:348] T 478014fd7c524f18b0f86961e6143256 P 4b26fa9b4a984025a159d0f38be81428 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b26fa9b4a984025a159d0f38be81428" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 41697 } } peers { permanent_uuid: "61f45f84da5f4e89bd2a27ed0cdc051b" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 40541 } } peers { permanent_uuid: "7e334a6d68eb4d98853ce401bd25949b" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 34229 } }
I20260806 04:57:25.544265 13014 raft_consensus.cc:374] T 478014fd7c524f18b0f86961e6143256 P 4b26fa9b4a984025a159d0f38be81428 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 04:57:25.544394 13014 raft_consensus.cc:729] T 478014fd7c524f18b0f86961e6143256 P 4b26fa9b4a984025a159d0f38be81428 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b26fa9b4a984025a159d0f38be81428, State: Initialized, Role: FOLLOWER
I20260806 04:57:25.544668 13014 consensus_queue.cc:260] T 478014fd7c524f18b0f86961e6143256 P 4b26fa9b4a984025a159d0f38be81428 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b26fa9b4a984025a159d0f38be81428" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 41697 } } peers { permanent_uuid: "61f45f84da5f4e89bd2a27ed0cdc051b" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 40541 } } peers { permanent_uuid: "7e334a6d68eb4d98853ce401bd25949b" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 34229 } }
I20260806 04:57:25.546164 13014 ts_tablet_manager.cc:1428] T 478014fd7c524f18b0f86961e6143256 P 4b26fa9b4a984025a159d0f38be81428: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260806 04:57:25.645476 12945 raft_consensus.cc:482] T 478014fd7c524f18b0f86961e6143256 P 4b26fa9b4a984025a159d0f38be81428 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 04:57:25.645709 12945 raft_consensus.cc:504] T 478014fd7c524f18b0f86961e6143256 P 4b26fa9b4a984025a159d0f38be81428 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b26fa9b4a984025a159d0f38be81428" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 41697 } } peers { permanent_uuid: "61f45f84da5f4e89bd2a27ed0cdc051b" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 40541 } } peers { permanent_uuid: "7e334a6d68eb4d98853ce401bd25949b" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 34229 } }
I20260806 04:57:25.646616 12945 leader_election.cc:288] T 478014fd7c524f18b0f86961e6143256 P 4b26fa9b4a984025a159d0f38be81428 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 61f45f84da5f4e89bd2a27ed0cdc051b (127.9.96.66:40541), 7e334a6d68eb4d98853ce401bd25949b (127.9.96.67:34229)
I20260806 04:57:25.647586 12640 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "478014fd7c524f18b0f86961e6143256" candidate_uuid: "4b26fa9b4a984025a159d0f38be81428" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "61f45f84da5f4e89bd2a27ed0cdc051b" is_pre_election: true
I20260806 04:57:25.647895 12640 raft_consensus.cc:2454] T 478014fd7c524f18b0f86961e6143256 P 61f45f84da5f4e89bd2a27ed0cdc051b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b26fa9b4a984025a159d0f38be81428 in term 0.
I20260806 04:57:25.648492 12336 leader_election.cc:302] T 478014fd7c524f18b0f86961e6143256 P 4b26fa9b4a984025a159d0f38be81428 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4b26fa9b4a984025a159d0f38be81428, 61f45f84da5f4e89bd2a27ed0cdc051b; no voters:
I20260806 04:57:25.649431 12845 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "478014fd7c524f18b0f86961e6143256" candidate_uuid: "4b26fa9b4a984025a159d0f38be81428" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e334a6d68eb4d98853ce401bd25949b" is_pre_election: true
I20260806 04:57:25.649719 12845 raft_consensus.cc:2454] T 478014fd7c524f18b0f86961e6143256 P 7e334a6d68eb4d98853ce401bd25949b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b26fa9b4a984025a159d0f38be81428 in term 0.
I20260806 04:57:25.650449 12945 raft_consensus.cc:2781] T 478014fd7c524f18b0f86961e6143256 P 4b26fa9b4a984025a159d0f38be81428 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 04:57:25.650589 12945 raft_consensus.cc:482] T 478014fd7c524f18b0f86961e6143256 P 4b26fa9b4a984025a159d0f38be81428 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 04:57:25.650705 12945 raft_consensus.cc:3037] T 478014fd7c524f18b0f86961e6143256 P 4b26fa9b4a984025a159d0f38be81428 [term 0 FOLLOWER]: Advancing to term 1
I20260806 04:57:25.654574 12945 raft_consensus.cc:504] T 478014fd7c524f18b0f86961e6143256 P 4b26fa9b4a984025a159d0f38be81428 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b26fa9b4a984025a159d0f38be81428" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 41697 } } peers { permanent_uuid: "61f45f84da5f4e89bd2a27ed0cdc051b" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 40541 } } peers { permanent_uuid: "7e334a6d68eb4d98853ce401bd25949b" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 34229 } }
I20260806 04:57:25.655889 12640 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "478014fd7c524f18b0f86961e6143256" candidate_uuid: "4b26fa9b4a984025a159d0f38be81428" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "61f45f84da5f4e89bd2a27ed0cdc051b"
I20260806 04:57:25.656126 12640 raft_consensus.cc:3037] T 478014fd7c524f18b0f86961e6143256 P 61f45f84da5f4e89bd2a27ed0cdc051b [term 0 FOLLOWER]: Advancing to term 1
I20260806 04:57:25.659518 12640 raft_consensus.cc:2454] T 478014fd7c524f18b0f86961e6143256 P 61f45f84da5f4e89bd2a27ed0cdc051b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b26fa9b4a984025a159d0f38be81428 in term 1.
I20260806 04:57:25.660022 12336 leader_election.cc:302] T 478014fd7c524f18b0f86961e6143256 P 4b26fa9b4a984025a159d0f38be81428 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4b26fa9b4a984025a159d0f38be81428, 61f45f84da5f4e89bd2a27ed0cdc051b; no voters:
I20260806 04:57:25.660382 12982 raft_consensus.cc:2781] T 478014fd7c524f18b0f86961e6143256 P 4b26fa9b4a984025a159d0f38be81428 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 04:57:25.660562 12982 raft_consensus.cc:686] T 478014fd7c524f18b0f86961e6143256 P 4b26fa9b4a984025a159d0f38be81428 [term 1 LEADER]: Becoming Leader. State: Replica: 4b26fa9b4a984025a159d0f38be81428, State: Running, Role: LEADER
I20260806 04:57:25.660890 12982 consensus_queue.cc:237] T 478014fd7c524f18b0f86961e6143256 P 4b26fa9b4a984025a159d0f38be81428 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b26fa9b4a984025a159d0f38be81428" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 41697 } } peers { permanent_uuid: "61f45f84da5f4e89bd2a27ed0cdc051b" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 40541 } } peers { permanent_uuid: "7e334a6d68eb4d98853ce401bd25949b" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 34229 } }
I20260806 04:57:25.661314 12845 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "478014fd7c524f18b0f86961e6143256" candidate_uuid: "4b26fa9b4a984025a159d0f38be81428" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e334a6d68eb4d98853ce401bd25949b"
I20260806 04:57:25.661548 12845 raft_consensus.cc:3037] T 478014fd7c524f18b0f86961e6143256 P 7e334a6d68eb4d98853ce401bd25949b [term 0 FOLLOWER]: Advancing to term 1
I20260806 04:57:25.672072 12845 raft_consensus.cc:2454] T 478014fd7c524f18b0f86961e6143256 P 7e334a6d68eb4d98853ce401bd25949b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b26fa9b4a984025a159d0f38be81428 in term 1.
I20260806 04:57:25.676131 12210 catalog_manager.cc:5495] T 478014fd7c524f18b0f86961e6143256 P 4b26fa9b4a984025a159d0f38be81428 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4b26fa9b4a984025a159d0f38be81428 (127.9.96.65). New cstate: current_term: 1 leader_uuid: "4b26fa9b4a984025a159d0f38be81428" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b26fa9b4a984025a159d0f38be81428" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 41697 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "61f45f84da5f4e89bd2a27ed0cdc051b" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 40541 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7e334a6d68eb4d98853ce401bd25949b" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 34229 } health_report { overall_health: UNKNOWN } } }
I20260806 04:57:25.677198 12945 leader_election.cc:288] T 478014fd7c524f18b0f86961e6143256 P 4b26fa9b4a984025a159d0f38be81428 [CANDIDATE]: Term 1 election: Requested vote from peers 61f45f84da5f4e89bd2a27ed0cdc051b (127.9.96.66:40541), 7e334a6d68eb4d98853ce401bd25949b (127.9.96.67:34229)
I20260806 04:57:25.730924 12934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:57:25.724492 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50040 (local address 127.9.96.65:41697)
0806 04:57:25.724648 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0806 04:57:25.724654 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 04:57:25.724768 (+ 114us) server_negotiation.cc:407] Connection header received
0806 04:57:25.724972 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:57:25.724977 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:57:25.725061 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:57:25.725152 (+ 91us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 04:57:25.725601 (+ 449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:57:25.726770 (+ 1169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:57:25.728757 (+ 1987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:57:25.729497 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:57:25.729640 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:57:25.730082 (+ 442us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 04:57:25.730230 (+ 148us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 04:57:25.730316 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0806 04:57:25.730519 (+ 203us) server_negotiation.cc:299] Negotiation successful
0806 04:57:25.730606 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260806 04:57:25.746372 12934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:57:25.740397 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.96.65:39075 (local address 127.9.96.65:41697)
0806 04:57:25.740559 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0806 04:57:25.740565 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 04:57:25.740581 (+ 16us) server_negotiation.cc:407] Connection header received
0806 04:57:25.740637 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:57:25.740641 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:57:25.740734 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:57:25.741162 (+ 428us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 04:57:25.741272 (+ 110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:57:25.742346 (+ 1074us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:57:25.744364 (+ 2018us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:57:25.745069 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:57:25.745190 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:57:25.745651 (+ 461us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 04:57:25.745797 (+ 146us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 04:57:25.745963 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0806 04:57:25.746014 (+ 51us) server_negotiation.cc:299] Negotiation successful
0806 04:57:25.746090 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260806 04:57:25.746461 12997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:57:25.739800 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.65:41697 (local address 127.9.96.65:39075)
0806 04:57:25.740069 (+ 269us) negotiation.cc:107] Waiting for socket to connect
0806 04:57:25.740088 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0806 04:57:25.740190 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:57:25.740850 (+ 660us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:57:25.740854 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:57:25.740866 (+ 12us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 04:57:25.741044 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:57:25.741053 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:57:25.742512 (+ 1459us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:57:25.742518 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:57:25.744212 (+ 1694us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:57:25.744228 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:57:25.745231 (+ 1003us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:57:25.745238 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:57:25.745434 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:57:25.745483 (+ 49us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 04:57:25.745913 (+ 430us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 04:57:25.745926 (+ 13us) client_negotiation.cc:769] Sending connection context
0806 04:57:25.746000 (+ 74us) client_negotiation.cc:240] Negotiation successful
0806 04:57:25.746140 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122}
I20260806 04:57:25.757040 12640 raft_consensus.cc:1261] T 478014fd7c524f18b0f86961e6143256 P 61f45f84da5f4e89bd2a27ed0cdc051b [term 1 FOLLOWER]: Refusing update from remote peer 4b26fa9b4a984025a159d0f38be81428: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 04:57:25.757052 12845 raft_consensus.cc:1261] T 478014fd7c524f18b0f86961e6143256 P 7e334a6d68eb4d98853ce401bd25949b [term 1 FOLLOWER]: Refusing update from remote peer 4b26fa9b4a984025a159d0f38be81428: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 04:57:25.757696 12945 consensus_queue.cc:1035] T 478014fd7c524f18b0f86961e6143256 P 4b26fa9b4a984025a159d0f38be81428 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e334a6d68eb4d98853ce401bd25949b" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 34229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 04:57:25.758045 12982 consensus_queue.cc:1035] T 478014fd7c524f18b0f86961e6143256 P 4b26fa9b4a984025a159d0f38be81428 [LEADER]: Connected to new peer: Peer: permanent_uuid: "61f45f84da5f4e89bd2a27ed0cdc051b" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 40541 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 04:57:25.762166 12971 mvcc.cc:205] Tried to move back new op lower bound from 7315424238616145920 to 7315424238229966848. Current Snapshot: MvccSnapshot[applied={T|T < 7315424238616145920}]
W20260806 04:57:25.783308 12971 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260806 04:57:27.382443 12208 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47864:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 04:57:27.382656 12208 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47864:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 04:57:27.386638 12208 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ebfef4bb062640d9bf626f0e62f6b98c: Sending DeleteTablet for 3 replicas of tablet 478014fd7c524f18b0f86961e6143256
I20260806 04:57:27.387488 12408 tablet_service.cc:1514] Processing DeleteTablet for tablet 478014fd7c524f18b0f86961e6143256 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 04:57:27 UTC) from {username='kudu', principal='kudu/127.9.96.126@KRBTEST.COM'} at 127.0.0.1:50012
I20260806 04:57:27.387926 12806 tablet_service.cc:1514] Processing DeleteTablet for tablet 478014fd7c524f18b0f86961e6143256 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 04:57:27 UTC) from {username='kudu', principal='kudu/127.9.96.126@KRBTEST.COM'} at 127.0.0.1:60914
I20260806 04:57:27.387952 12608 tablet_service.cc:1514] Processing DeleteTablet for tablet 478014fd7c524f18b0f86961e6143256 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 04:57:27 UTC) from {username='kudu', principal='kudu/127.9.96.126@KRBTEST.COM'} at 127.0.0.1:44726
I20260806 04:57:27.403205 13050 tablet_replica.cc:330] T 478014fd7c524f18b0f86961e6143256 P 4b26fa9b4a984025a159d0f38be81428: stopping tablet replica
I20260806 04:57:27.403406 13052 tablet_replica.cc:330] T 478014fd7c524f18b0f86961e6143256 P 61f45f84da5f4e89bd2a27ed0cdc051b: stopping tablet replica
I20260806 04:57:27.403470 13050 raft_consensus.cc:2229] T 478014fd7c524f18b0f86961e6143256 P 4b26fa9b4a984025a159d0f38be81428 [term 1 LEADER]: Raft consensus shutting down.
I20260806 04:57:27.403601 13052 raft_consensus.cc:2229] T 478014fd7c524f18b0f86961e6143256 P 61f45f84da5f4e89bd2a27ed0cdc051b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 04:57:27.403851 13052 raft_consensus.cc:2258] T 478014fd7c524f18b0f86961e6143256 P 61f45f84da5f4e89bd2a27ed0cdc051b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 04:57:27.404101 13050 raft_consensus.cc:2258] T 478014fd7c524f18b0f86961e6143256 P 4b26fa9b4a984025a159d0f38be81428 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 04:57:27.404495 13051 tablet_replica.cc:330] T 478014fd7c524f18b0f86961e6143256 P 7e334a6d68eb4d98853ce401bd25949b: stopping tablet replica
I20260806 04:57:27.404700 13051 raft_consensus.cc:2229] T 478014fd7c524f18b0f86961e6143256 P 7e334a6d68eb4d98853ce401bd25949b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 04:57:27.404968 13051 raft_consensus.cc:2258] T 478014fd7c524f18b0f86961e6143256 P 7e334a6d68eb4d98853ce401bd25949b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 04:57:27.405149 13052 ts_tablet_manager.cc:1905] T 478014fd7c524f18b0f86961e6143256 P 61f45f84da5f4e89bd2a27ed0cdc051b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 04:57:27.405777 13050 ts_tablet_manager.cc:1905] T 478014fd7c524f18b0f86961e6143256 P 4b26fa9b4a984025a159d0f38be81428: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 04:57:27.406163 13051 ts_tablet_manager.cc:1905] T 478014fd7c524f18b0f86961e6143256 P 7e334a6d68eb4d98853ce401bd25949b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 04:57:27.411680 13052 ts_tablet_manager.cc:1918] T 478014fd7c524f18b0f86961e6143256 P 61f45f84da5f4e89bd2a27ed0cdc051b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260806 04:57:27.411819 13052 log.cc:1196] T 478014fd7c524f18b0f86961e6143256 P 61f45f84da5f4e89bd2a27ed0cdc051b: Deleting WAL directory at /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785992227050689-9601-0/minicluster-data/ts-1/wal/wals/478014fd7c524f18b0f86961e6143256
I20260806 04:57:27.412207 13050 ts_tablet_manager.cc:1918] T 478014fd7c524f18b0f86961e6143256 P 4b26fa9b4a984025a159d0f38be81428: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260806 04:57:27.412312 13052 ts_tablet_manager.cc:1939] T 478014fd7c524f18b0f86961e6143256 P 61f45f84da5f4e89bd2a27ed0cdc051b: Deleting consensus metadata
I20260806 04:57:27.412370 13050 log.cc:1196] T 478014fd7c524f18b0f86961e6143256 P 4b26fa9b4a984025a159d0f38be81428: Deleting WAL directory at /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785992227050689-9601-0/minicluster-data/ts-0/wal/wals/478014fd7c524f18b0f86961e6143256
I20260806 04:57:27.412822 13050 ts_tablet_manager.cc:1939] T 478014fd7c524f18b0f86961e6143256 P 4b26fa9b4a984025a159d0f38be81428: Deleting consensus metadata
I20260806 04:57:27.413663 12185 catalog_manager.cc:4841] TS 61f45f84da5f4e89bd2a27ed0cdc051b (127.9.96.66:40541): tablet 478014fd7c524f18b0f86961e6143256 (table test-table [id=fec0c04e05b74f78a6328c5bc1eed5c0]) successfully deleted
I20260806 04:57:27.414034 12187 catalog_manager.cc:4841] TS 4b26fa9b4a984025a159d0f38be81428 (127.9.96.65:41697): tablet 478014fd7c524f18b0f86961e6143256 (table test-table [id=fec0c04e05b74f78a6328c5bc1eed5c0]) successfully deleted
I20260806 04:57:27.414072 13051 ts_tablet_manager.cc:1918] T 478014fd7c524f18b0f86961e6143256 P 7e334a6d68eb4d98853ce401bd25949b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260806 04:57:27.414213 13051 log.cc:1196] T 478014fd7c524f18b0f86961e6143256 P 7e334a6d68eb4d98853ce401bd25949b: Deleting WAL directory at /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785992227050689-9601-0/minicluster-data/ts-2/wal/wals/478014fd7c524f18b0f86961e6143256
I20260806 04:57:27.414701 13051 ts_tablet_manager.cc:1939] T 478014fd7c524f18b0f86961e6143256 P 7e334a6d68eb4d98853ce401bd25949b: Deleting consensus metadata
I20260806 04:57:27.415704 12187 catalog_manager.cc:4841] TS 7e334a6d68eb4d98853ce401bd25949b (127.9.96.67:34229): tablet 478014fd7c524f18b0f86961e6143256 (table test-table [id=fec0c04e05b74f78a6328c5bc1eed5c0]) successfully deleted
Aug 06 04:57:27 dist-test-slave-x4qh krb5kdc[12131](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992247, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 06 04:57:27 dist-test-slave-x4qh krb5kdc[12131](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992247, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.9.96.126@KRBTEST.COM
I20260806 04:57:27.454608 13067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:57:27.436509 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47870 (local address 127.9.96.126:39063)
0806 04:57:27.437157 (+ 648us) server_negotiation.cc:206] Beginning negotiation
0806 04:57:27.437162 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 04:57:27.437184 (+ 22us) server_negotiation.cc:407] Connection header received
0806 04:57:27.437369 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:57:27.437374 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:57:27.437457 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:57:27.437565 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:57:27.439309 (+ 1744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:57:27.440174 (+ 865us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:57:27.441405 (+ 1231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:57:27.441806 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:57:27.441902 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:57:27.449106 (+ 7204us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:57:27.449111 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:57:27.449114 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:57:27.449148 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:57:27.452129 (+ 2981us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:57:27.452820 (+ 691us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:57:27.452824 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:57:27.452827 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:57:27.452954 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:57:27.453312 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:57:27.453315 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:57:27.453317 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:57:27.453581 (+ 264us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:57:27.453757 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0806 04:57:27.454057 (+ 300us) server_negotiation.cc:299] Negotiation successful
0806 04:57:27.454220 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":521,"thread_start_us":329,"threads_started":1}
I20260806 04:57:27.458117 12208 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47870:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 04:57:27.458659 12208 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 04:57:27.464999 12408 tablet_service.cc:1467] Processing CreateTablet for tablet 0ba5ac7e0e984652a0dc6c807c46e33a (DEFAULT_TABLE table=test-table [id=cb0eb0cbcdaa43a0aafe639bc705c826]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 04:57:27.465063 12608 tablet_service.cc:1467] Processing CreateTablet for tablet 0ba5ac7e0e984652a0dc6c807c46e33a (DEFAULT_TABLE table=test-table [id=cb0eb0cbcdaa43a0aafe639bc705c826]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 04:57:27.465490 12408 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0ba5ac7e0e984652a0dc6c807c46e33a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 04:57:27.465517 12608 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0ba5ac7e0e984652a0dc6c807c46e33a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 04:57:27.465739 12806 tablet_service.cc:1467] Processing CreateTablet for tablet 0ba5ac7e0e984652a0dc6c807c46e33a (DEFAULT_TABLE table=test-table [id=cb0eb0cbcdaa43a0aafe639bc705c826]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 04:57:27.466230 12806 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0ba5ac7e0e984652a0dc6c807c46e33a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 04:57:27.471937 13069 tablet_bootstrap.cc:492] T 0ba5ac7e0e984652a0dc6c807c46e33a P 7e334a6d68eb4d98853ce401bd25949b: Bootstrap starting.
I20260806 04:57:27.473686 13069 tablet_bootstrap.cc:654] T 0ba5ac7e0e984652a0dc6c807c46e33a P 7e334a6d68eb4d98853ce401bd25949b: Neither blocks nor log segments found. Creating new log.
I20260806 04:57:27.475924 13069 tablet_bootstrap.cc:492] T 0ba5ac7e0e984652a0dc6c807c46e33a P 7e334a6d68eb4d98853ce401bd25949b: No bootstrap required, opened a new log
I20260806 04:57:27.476073 13069 ts_tablet_manager.cc:1397] T 0ba5ac7e0e984652a0dc6c807c46e33a P 7e334a6d68eb4d98853ce401bd25949b: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.003s
I20260806 04:57:27.476707 13069 raft_consensus.cc:348] T 0ba5ac7e0e984652a0dc6c807c46e33a P 7e334a6d68eb4d98853ce401bd25949b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b26fa9b4a984025a159d0f38be81428" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 41697 } } peers { permanent_uuid: "61f45f84da5f4e89bd2a27ed0cdc051b" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 40541 } } peers { permanent_uuid: "7e334a6d68eb4d98853ce401bd25949b" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 34229 } }
I20260806 04:57:27.476886 13069 raft_consensus.cc:374] T 0ba5ac7e0e984652a0dc6c807c46e33a P 7e334a6d68eb4d98853ce401bd25949b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 04:57:27.477008 13069 raft_consensus.cc:729] T 0ba5ac7e0e984652a0dc6c807c46e33a P 7e334a6d68eb4d98853ce401bd25949b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e334a6d68eb4d98853ce401bd25949b, State: Initialized, Role: FOLLOWER
I20260806 04:57:27.477274 13069 consensus_queue.cc:260] T 0ba5ac7e0e984652a0dc6c807c46e33a P 7e334a6d68eb4d98853ce401bd25949b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b26fa9b4a984025a159d0f38be81428" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 41697 } } peers { permanent_uuid: "61f45f84da5f4e89bd2a27ed0cdc051b" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 40541 } } peers { permanent_uuid: "7e334a6d68eb4d98853ce401bd25949b" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 34229 } }
I20260806 04:57:27.478142 13069 ts_tablet_manager.cc:1428] T 0ba5ac7e0e984652a0dc6c807c46e33a P 7e334a6d68eb4d98853ce401bd25949b: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260806 04:57:27.478803 13071 tablet_bootstrap.cc:492] T 0ba5ac7e0e984652a0dc6c807c46e33a P 4b26fa9b4a984025a159d0f38be81428: Bootstrap starting.
I20260806 04:57:27.479452 13073 tablet_bootstrap.cc:492] T 0ba5ac7e0e984652a0dc6c807c46e33a P 61f45f84da5f4e89bd2a27ed0cdc051b: Bootstrap starting.
I20260806 04:57:27.480564 13071 tablet_bootstrap.cc:654] T 0ba5ac7e0e984652a0dc6c807c46e33a P 4b26fa9b4a984025a159d0f38be81428: Neither blocks nor log segments found. Creating new log.
I20260806 04:57:27.481060 13073 tablet_bootstrap.cc:654] T 0ba5ac7e0e984652a0dc6c807c46e33a P 61f45f84da5f4e89bd2a27ed0cdc051b: Neither blocks nor log segments found. Creating new log.
I20260806 04:57:27.482874 13071 tablet_bootstrap.cc:492] T 0ba5ac7e0e984652a0dc6c807c46e33a P 4b26fa9b4a984025a159d0f38be81428: No bootstrap required, opened a new log
I20260806 04:57:27.483031 13071 ts_tablet_manager.cc:1397] T 0ba5ac7e0e984652a0dc6c807c46e33a P 4b26fa9b4a984025a159d0f38be81428: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260806 04:57:27.483862 13071 raft_consensus.cc:348] T 0ba5ac7e0e984652a0dc6c807c46e33a P 4b26fa9b4a984025a159d0f38be81428 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b26fa9b4a984025a159d0f38be81428" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 41697 } } peers { permanent_uuid: "61f45f84da5f4e89bd2a27ed0cdc051b" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 40541 } } peers { permanent_uuid: "7e334a6d68eb4d98853ce401bd25949b" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 34229 } }
I20260806 04:57:27.484077 13071 raft_consensus.cc:374] T 0ba5ac7e0e984652a0dc6c807c46e33a P 4b26fa9b4a984025a159d0f38be81428 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 04:57:27.484179 13071 raft_consensus.cc:729] T 0ba5ac7e0e984652a0dc6c807c46e33a P 4b26fa9b4a984025a159d0f38be81428 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b26fa9b4a984025a159d0f38be81428, State: Initialized, Role: FOLLOWER
I20260806 04:57:27.484464 13071 consensus_queue.cc:260] T 0ba5ac7e0e984652a0dc6c807c46e33a P 4b26fa9b4a984025a159d0f38be81428 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b26fa9b4a984025a159d0f38be81428" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 41697 } } peers { permanent_uuid: "61f45f84da5f4e89bd2a27ed0cdc051b" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 40541 } } peers { permanent_uuid: "7e334a6d68eb4d98853ce401bd25949b" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 34229 } }
I20260806 04:57:27.484985 13071 ts_tablet_manager.cc:1428] T 0ba5ac7e0e984652a0dc6c807c46e33a P 4b26fa9b4a984025a159d0f38be81428: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260806 04:57:27.486474 13073 tablet_bootstrap.cc:492] T 0ba5ac7e0e984652a0dc6c807c46e33a P 61f45f84da5f4e89bd2a27ed0cdc051b: No bootstrap required, opened a new log
I20260806 04:57:27.486635 13073 ts_tablet_manager.cc:1397] T 0ba5ac7e0e984652a0dc6c807c46e33a P 61f45f84da5f4e89bd2a27ed0cdc051b: Time spent bootstrapping tablet: real 0.011s user 0.006s sys 0.000s
I20260806 04:57:27.487457 13073 raft_consensus.cc:348] T 0ba5ac7e0e984652a0dc6c807c46e33a P 61f45f84da5f4e89bd2a27ed0cdc051b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b26fa9b4a984025a159d0f38be81428" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 41697 } } peers { permanent_uuid: "61f45f84da5f4e89bd2a27ed0cdc051b" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 40541 } } peers { permanent_uuid: "7e334a6d68eb4d98853ce401bd25949b" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 34229 } }
I20260806 04:57:27.487718 13073 raft_consensus.cc:374] T 0ba5ac7e0e984652a0dc6c807c46e33a P 61f45f84da5f4e89bd2a27ed0cdc051b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 04:57:27.487824 13073 raft_consensus.cc:729] T 0ba5ac7e0e984652a0dc6c807c46e33a P 61f45f84da5f4e89bd2a27ed0cdc051b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 61f45f84da5f4e89bd2a27ed0cdc051b, State: Initialized, Role: FOLLOWER
I20260806 04:57:27.488085 13073 consensus_queue.cc:260] T 0ba5ac7e0e984652a0dc6c807c46e33a P 61f45f84da5f4e89bd2a27ed0cdc051b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b26fa9b4a984025a159d0f38be81428" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 41697 } } peers { permanent_uuid: "61f45f84da5f4e89bd2a27ed0cdc051b" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 40541 } } peers { permanent_uuid: "7e334a6d68eb4d98853ce401bd25949b" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 34229 } }
I20260806 04:57:27.489012 13073 ts_tablet_manager.cc:1428] T 0ba5ac7e0e984652a0dc6c807c46e33a P 61f45f84da5f4e89bd2a27ed0cdc051b: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260806 04:57:27.683672 13082 raft_consensus.cc:482] T 0ba5ac7e0e984652a0dc6c807c46e33a P 61f45f84da5f4e89bd2a27ed0cdc051b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 04:57:27.683895 13082 raft_consensus.cc:504] T 0ba5ac7e0e984652a0dc6c807c46e33a P 61f45f84da5f4e89bd2a27ed0cdc051b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b26fa9b4a984025a159d0f38be81428" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 41697 } } peers { permanent_uuid: "61f45f84da5f4e89bd2a27ed0cdc051b" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 40541 } } peers { permanent_uuid: "7e334a6d68eb4d98853ce401bd25949b" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 34229 } }
I20260806 04:57:27.684693 13082 leader_election.cc:288] T 0ba5ac7e0e984652a0dc6c807c46e33a P 61f45f84da5f4e89bd2a27ed0cdc051b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4b26fa9b4a984025a159d0f38be81428 (127.9.96.65:41697), 7e334a6d68eb4d98853ce401bd25949b (127.9.96.67:34229)
I20260806 04:57:27.685350 12441 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0ba5ac7e0e984652a0dc6c807c46e33a" candidate_uuid: "61f45f84da5f4e89bd2a27ed0cdc051b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b26fa9b4a984025a159d0f38be81428" is_pre_election: true
I20260806 04:57:27.685348 12845 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0ba5ac7e0e984652a0dc6c807c46e33a" candidate_uuid: "61f45f84da5f4e89bd2a27ed0cdc051b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e334a6d68eb4d98853ce401bd25949b" is_pre_election: true
I20260806 04:57:27.685624 12845 raft_consensus.cc:2454] T 0ba5ac7e0e984652a0dc6c807c46e33a P 7e334a6d68eb4d98853ce401bd25949b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 61f45f84da5f4e89bd2a27ed0cdc051b in term 0.
I20260806 04:57:27.685623 12441 raft_consensus.cc:2454] T 0ba5ac7e0e984652a0dc6c807c46e33a P 4b26fa9b4a984025a159d0f38be81428 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 61f45f84da5f4e89bd2a27ed0cdc051b in term 0.
I20260806 04:57:27.686103 12544 leader_election.cc:302] T 0ba5ac7e0e984652a0dc6c807c46e33a P 61f45f84da5f4e89bd2a27ed0cdc051b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4b26fa9b4a984025a159d0f38be81428, 61f45f84da5f4e89bd2a27ed0cdc051b; no voters:
I20260806 04:57:27.686367 13082 raft_consensus.cc:2781] T 0ba5ac7e0e984652a0dc6c807c46e33a P 61f45f84da5f4e89bd2a27ed0cdc051b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 04:57:27.686496 13082 raft_consensus.cc:482] T 0ba5ac7e0e984652a0dc6c807c46e33a P 61f45f84da5f4e89bd2a27ed0cdc051b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 04:57:27.686595 13082 raft_consensus.cc:3037] T 0ba5ac7e0e984652a0dc6c807c46e33a P 61f45f84da5f4e89bd2a27ed0cdc051b [term 0 FOLLOWER]: Advancing to term 1
I20260806 04:57:27.688879 13082 raft_consensus.cc:504] T 0ba5ac7e0e984652a0dc6c807c46e33a P 61f45f84da5f4e89bd2a27ed0cdc051b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b26fa9b4a984025a159d0f38be81428" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 41697 } } peers { permanent_uuid: "61f45f84da5f4e89bd2a27ed0cdc051b" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 40541 } } peers { permanent_uuid: "7e334a6d68eb4d98853ce401bd25949b" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 34229 } }
I20260806 04:57:27.689376 13082 leader_election.cc:288] T 0ba5ac7e0e984652a0dc6c807c46e33a P 61f45f84da5f4e89bd2a27ed0cdc051b [CANDIDATE]: Term 1 election: Requested vote from peers 4b26fa9b4a984025a159d0f38be81428 (127.9.96.65:41697), 7e334a6d68eb4d98853ce401bd25949b (127.9.96.67:34229)
I20260806 04:57:27.689865 12441 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0ba5ac7e0e984652a0dc6c807c46e33a" candidate_uuid: "61f45f84da5f4e89bd2a27ed0cdc051b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b26fa9b4a984025a159d0f38be81428"
I20260806 04:57:27.690050 12441 raft_consensus.cc:3037] T 0ba5ac7e0e984652a0dc6c807c46e33a P 4b26fa9b4a984025a159d0f38be81428 [term 0 FOLLOWER]: Advancing to term 1
I20260806 04:57:27.690009 12845 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0ba5ac7e0e984652a0dc6c807c46e33a" candidate_uuid: "61f45f84da5f4e89bd2a27ed0cdc051b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e334a6d68eb4d98853ce401bd25949b"
I20260806 04:57:27.690231 12845 raft_consensus.cc:3037] T 0ba5ac7e0e984652a0dc6c807c46e33a P 7e334a6d68eb4d98853ce401bd25949b [term 0 FOLLOWER]: Advancing to term 1
I20260806 04:57:27.692199 12441 raft_consensus.cc:2454] T 0ba5ac7e0e984652a0dc6c807c46e33a P 4b26fa9b4a984025a159d0f38be81428 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 61f45f84da5f4e89bd2a27ed0cdc051b in term 1.
I20260806 04:57:27.692390 12845 raft_consensus.cc:2454] T 0ba5ac7e0e984652a0dc6c807c46e33a P 7e334a6d68eb4d98853ce401bd25949b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 61f45f84da5f4e89bd2a27ed0cdc051b in term 1.
I20260806 04:57:27.692655 12544 leader_election.cc:302] T 0ba5ac7e0e984652a0dc6c807c46e33a P 61f45f84da5f4e89bd2a27ed0cdc051b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4b26fa9b4a984025a159d0f38be81428, 61f45f84da5f4e89bd2a27ed0cdc051b; no voters:
I20260806 04:57:27.692943 13082 raft_consensus.cc:2781] T 0ba5ac7e0e984652a0dc6c807c46e33a P 61f45f84da5f4e89bd2a27ed0cdc051b [term 1 FOLLOWER]: Leader election won for term 1
I20260806 04:57:27.693408 13082 raft_consensus.cc:686] T 0ba5ac7e0e984652a0dc6c807c46e33a P 61f45f84da5f4e89bd2a27ed0cdc051b [term 1 LEADER]: Becoming Leader. State: Replica: 61f45f84da5f4e89bd2a27ed0cdc051b, State: Running, Role: LEADER
I20260806 04:57:27.693734 13082 consensus_queue.cc:237] T 0ba5ac7e0e984652a0dc6c807c46e33a P 61f45f84da5f4e89bd2a27ed0cdc051b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b26fa9b4a984025a159d0f38be81428" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 41697 } } peers { permanent_uuid: "61f45f84da5f4e89bd2a27ed0cdc051b" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 40541 } } peers { permanent_uuid: "7e334a6d68eb4d98853ce401bd25949b" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 34229 } }
I20260806 04:57:27.696313 12208 catalog_manager.cc:5495] T 0ba5ac7e0e984652a0dc6c807c46e33a P 61f45f84da5f4e89bd2a27ed0cdc051b reported cstate change: term changed from 0 to 1, leader changed from <none> to 61f45f84da5f4e89bd2a27ed0cdc051b (127.9.96.66). New cstate: current_term: 1 leader_uuid: "61f45f84da5f4e89bd2a27ed0cdc051b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b26fa9b4a984025a159d0f38be81428" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 41697 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "61f45f84da5f4e89bd2a27ed0cdc051b" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 40541 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7e334a6d68eb4d98853ce401bd25949b" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 34229 } health_report { overall_health: UNKNOWN } } }
I20260806 04:57:27.718473 13087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:57:27.712799 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44750 (local address 127.9.96.66:40541)
0806 04:57:27.713482 (+ 683us) server_negotiation.cc:206] Beginning negotiation
0806 04:57:27.713487 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 04:57:27.713505 (+ 18us) server_negotiation.cc:407] Connection header received
0806 04:57:27.713553 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:57:27.713556 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:57:27.713612 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:57:27.713696 (+ 84us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 04:57:27.714270 (+ 574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:57:27.715362 (+ 1092us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:57:27.716761 (+ 1399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:57:27.717153 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:57:27.717244 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:57:27.717783 (+ 539us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 04:57:27.717891 (+ 108us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 04:57:27.717959 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0806 04:57:27.718135 (+ 176us) server_negotiation.cc:299] Negotiation successful
0806 04:57:27.718194 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":571,"thread_start_us":430,"threads_started":1}
I20260806 04:57:27.727662 12845 raft_consensus.cc:1261] T 0ba5ac7e0e984652a0dc6c807c46e33a P 7e334a6d68eb4d98853ce401bd25949b [term 1 FOLLOWER]: Refusing update from remote peer 61f45f84da5f4e89bd2a27ed0cdc051b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 04:57:27.727799 12441 raft_consensus.cc:1261] T 0ba5ac7e0e984652a0dc6c807c46e33a P 4b26fa9b4a984025a159d0f38be81428 [term 1 FOLLOWER]: Refusing update from remote peer 61f45f84da5f4e89bd2a27ed0cdc051b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 04:57:27.728266 13083 consensus_queue.cc:1035] T 0ba5ac7e0e984652a0dc6c807c46e33a P 61f45f84da5f4e89bd2a27ed0cdc051b [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e334a6d68eb4d98853ce401bd25949b" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 34229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 04:57:27.728556 13082 consensus_queue.cc:1035] T 0ba5ac7e0e984652a0dc6c807c46e33a P 61f45f84da5f4e89bd2a27ed0cdc051b [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b26fa9b4a984025a159d0f38be81428" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 41697 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 04:57:27.772567 12208 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47870:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 04:57:27.772866 12208 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47870:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 04:57:27.775725 12208 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ebfef4bb062640d9bf626f0e62f6b98c: Sending DeleteTablet for 3 replicas of tablet 0ba5ac7e0e984652a0dc6c807c46e33a
I20260806 04:57:27.776533 12608 tablet_service.cc:1514] Processing DeleteTablet for tablet 0ba5ac7e0e984652a0dc6c807c46e33a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 04:57:27 UTC) from {username='kudu', principal='kudu/127.9.96.126@KRBTEST.COM'} at 127.0.0.1:44726
I20260806 04:57:27.776567 12408 tablet_service.cc:1514] Processing DeleteTablet for tablet 0ba5ac7e0e984652a0dc6c807c46e33a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 04:57:27 UTC) from {username='kudu', principal='kudu/127.9.96.126@KRBTEST.COM'} at 127.0.0.1:50012
I20260806 04:57:27.776576 12806 tablet_service.cc:1514] Processing DeleteTablet for tablet 0ba5ac7e0e984652a0dc6c807c46e33a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 04:57:27 UTC) from {username='kudu', principal='kudu/127.9.96.126@KRBTEST.COM'} at 127.0.0.1:60914
I20260806 04:57:27.776844 13052 tablet_replica.cc:330] T 0ba5ac7e0e984652a0dc6c807c46e33a P 61f45f84da5f4e89bd2a27ed0cdc051b: stopping tablet replica
I20260806 04:57:27.776844 13050 tablet_replica.cc:330] T 0ba5ac7e0e984652a0dc6c807c46e33a P 4b26fa9b4a984025a159d0f38be81428: stopping tablet replica
I20260806 04:57:27.776854 13051 tablet_replica.cc:330] T 0ba5ac7e0e984652a0dc6c807c46e33a P 7e334a6d68eb4d98853ce401bd25949b: stopping tablet replica
I20260806 04:57:27.777046 13052 raft_consensus.cc:2229] T 0ba5ac7e0e984652a0dc6c807c46e33a P 61f45f84da5f4e89bd2a27ed0cdc051b [term 1 LEADER]: Raft consensus shutting down.
I20260806 04:57:27.777046 13050 raft_consensus.cc:2229] T 0ba5ac7e0e984652a0dc6c807c46e33a P 4b26fa9b4a984025a159d0f38be81428 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 04:57:27.777062 13051 raft_consensus.cc:2229] T 0ba5ac7e0e984652a0dc6c807c46e33a P 7e334a6d68eb4d98853ce401bd25949b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 04:57:27.777225 13050 raft_consensus.cc:2258] T 0ba5ac7e0e984652a0dc6c807c46e33a P 4b26fa9b4a984025a159d0f38be81428 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 04:57:27.777256 13051 raft_consensus.cc:2258] T 0ba5ac7e0e984652a0dc6c807c46e33a P 7e334a6d68eb4d98853ce401bd25949b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 04:57:27.777345 13052 raft_consensus.cc:2258] T 0ba5ac7e0e984652a0dc6c807c46e33a P 61f45f84da5f4e89bd2a27ed0cdc051b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 04:57:27.778353 13050 ts_tablet_manager.cc:1905] T 0ba5ac7e0e984652a0dc6c807c46e33a P 4b26fa9b4a984025a159d0f38be81428: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 04:57:27.778471 13051 ts_tablet_manager.cc:1905] T 0ba5ac7e0e984652a0dc6c807c46e33a P 7e334a6d68eb4d98853ce401bd25949b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 04:57:27.778488 13052 ts_tablet_manager.cc:1905] T 0ba5ac7e0e984652a0dc6c807c46e33a P 61f45f84da5f4e89bd2a27ed0cdc051b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 04:57:27.778352 12208 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47870:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 04:57:27.778975 12208 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 04:57:27.785173 13051 ts_tablet_manager.cc:1918] T 0ba5ac7e0e984652a0dc6c807c46e33a P 7e334a6d68eb4d98853ce401bd25949b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 04:57:27.785380 13051 log.cc:1196] T 0ba5ac7e0e984652a0dc6c807c46e33a P 7e334a6d68eb4d98853ce401bd25949b: Deleting WAL directory at /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785992227050689-9601-0/minicluster-data/ts-2/wal/wals/0ba5ac7e0e984652a0dc6c807c46e33a
I20260806 04:57:27.785893 13051 ts_tablet_manager.cc:1939] T 0ba5ac7e0e984652a0dc6c807c46e33a P 7e334a6d68eb4d98853ce401bd25949b: Deleting consensus metadata
I20260806 04:57:27.785822 12408 tablet_service.cc:1467] Processing CreateTablet for tablet bfc13521019948d0b8af5deee7e0d0af (DEFAULT_TABLE table=test-table [id=ac7824ff882d462d94377ab3bca6cbfc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 04:57:27.785910 12608 tablet_service.cc:1467] Processing CreateTablet for tablet bfc13521019948d0b8af5deee7e0d0af (DEFAULT_TABLE table=test-table [id=ac7824ff882d462d94377ab3bca6cbfc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 04:57:27.786265 12408 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bfc13521019948d0b8af5deee7e0d0af. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 04:57:27.786319 12608 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bfc13521019948d0b8af5deee7e0d0af. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 04:57:27.786485 13050 ts_tablet_manager.cc:1918] T 0ba5ac7e0e984652a0dc6c807c46e33a P 4b26fa9b4a984025a159d0f38be81428: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 04:57:27.786670 13050 log.cc:1196] T 0ba5ac7e0e984652a0dc6c807c46e33a P 4b26fa9b4a984025a159d0f38be81428: Deleting WAL directory at /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785992227050689-9601-0/minicluster-data/ts-0/wal/wals/0ba5ac7e0e984652a0dc6c807c46e33a
I20260806 04:57:27.787434 13050 ts_tablet_manager.cc:1939] T 0ba5ac7e0e984652a0dc6c807c46e33a P 4b26fa9b4a984025a159d0f38be81428: Deleting consensus metadata
I20260806 04:57:27.788252 12187 catalog_manager.cc:4841] TS 7e334a6d68eb4d98853ce401bd25949b (127.9.96.67:34229): tablet 0ba5ac7e0e984652a0dc6c807c46e33a (table test-table [id=cb0eb0cbcdaa43a0aafe639bc705c826]) successfully deleted
I20260806 04:57:27.788632 12187 catalog_manager.cc:4841] TS 4b26fa9b4a984025a159d0f38be81428 (127.9.96.65:41697): tablet 0ba5ac7e0e984652a0dc6c807c46e33a (table test-table [id=cb0eb0cbcdaa43a0aafe639bc705c826]) successfully deleted
I20260806 04:57:27.788600 12806 tablet_service.cc:1467] Processing CreateTablet for tablet bfc13521019948d0b8af5deee7e0d0af (DEFAULT_TABLE table=test-table [id=ac7824ff882d462d94377ab3bca6cbfc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 04:57:27.789011 12806 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bfc13521019948d0b8af5deee7e0d0af. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 04:57:27.792106 13073 tablet_bootstrap.cc:492] T bfc13521019948d0b8af5deee7e0d0af P 61f45f84da5f4e89bd2a27ed0cdc051b: Bootstrap starting.
I20260806 04:57:27.792655 13052 ts_tablet_manager.cc:1918] T 0ba5ac7e0e984652a0dc6c807c46e33a P 61f45f84da5f4e89bd2a27ed0cdc051b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 04:57:27.792937 13052 log.cc:1196] T 0ba5ac7e0e984652a0dc6c807c46e33a P 61f45f84da5f4e89bd2a27ed0cdc051b: Deleting WAL directory at /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785992227050689-9601-0/minicluster-data/ts-1/wal/wals/0ba5ac7e0e984652a0dc6c807c46e33a
I20260806 04:57:27.793488 13052 ts_tablet_manager.cc:1939] T 0ba5ac7e0e984652a0dc6c807c46e33a P 61f45f84da5f4e89bd2a27ed0cdc051b: Deleting consensus metadata
I20260806 04:57:27.793726 13071 tablet_bootstrap.cc:492] T bfc13521019948d0b8af5deee7e0d0af P 4b26fa9b4a984025a159d0f38be81428: Bootstrap starting.
I20260806 04:57:27.793960 13073 tablet_bootstrap.cc:654] T bfc13521019948d0b8af5deee7e0d0af P 61f45f84da5f4e89bd2a27ed0cdc051b: Neither blocks nor log segments found. Creating new log.
I20260806 04:57:27.795130 13069 tablet_bootstrap.cc:492] T bfc13521019948d0b8af5deee7e0d0af P 7e334a6d68eb4d98853ce401bd25949b: Bootstrap starting.
I20260806 04:57:27.795327 13071 tablet_bootstrap.cc:654] T bfc13521019948d0b8af5deee7e0d0af P 4b26fa9b4a984025a159d0f38be81428: Neither blocks nor log segments found. Creating new log.
I20260806 04:57:27.796176 12185 catalog_manager.cc:4841] TS 61f45f84da5f4e89bd2a27ed0cdc051b (127.9.96.66:40541): tablet 0ba5ac7e0e984652a0dc6c807c46e33a (table test-table [id=cb0eb0cbcdaa43a0aafe639bc705c826]) successfully deleted
I20260806 04:57:27.796976 13069 tablet_bootstrap.cc:654] T bfc13521019948d0b8af5deee7e0d0af P 7e334a6d68eb4d98853ce401bd25949b: Neither blocks nor log segments found. Creating new log.
I20260806 04:57:27.798309 13071 tablet_bootstrap.cc:492] T bfc13521019948d0b8af5deee7e0d0af P 4b26fa9b4a984025a159d0f38be81428: No bootstrap required, opened a new log
I20260806 04:57:27.798470 13071 ts_tablet_manager.cc:1397] T bfc13521019948d0b8af5deee7e0d0af P 4b26fa9b4a984025a159d0f38be81428: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.002s
I20260806 04:57:27.798888 13073 tablet_bootstrap.cc:492] T bfc13521019948d0b8af5deee7e0d0af P 61f45f84da5f4e89bd2a27ed0cdc051b: No bootstrap required, opened a new log
I20260806 04:57:27.799073 13073 ts_tablet_manager.cc:1397] T bfc13521019948d0b8af5deee7e0d0af P 61f45f84da5f4e89bd2a27ed0cdc051b: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.005s
I20260806 04:57:27.799280 13069 tablet_bootstrap.cc:492] T bfc13521019948d0b8af5deee7e0d0af P 7e334a6d68eb4d98853ce401bd25949b: No bootstrap required, opened a new log
I20260806 04:57:27.799491 13069 ts_tablet_manager.cc:1397] T bfc13521019948d0b8af5deee7e0d0af P 7e334a6d68eb4d98853ce401bd25949b: Time spent bootstrapping tablet: real 0.004s user 0.004s sys 0.000s
I20260806 04:57:27.799322 13071 raft_consensus.cc:348] T bfc13521019948d0b8af5deee7e0d0af P 4b26fa9b4a984025a159d0f38be81428 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b26fa9b4a984025a159d0f38be81428" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 41697 } } peers { permanent_uuid: "61f45f84da5f4e89bd2a27ed0cdc051b" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 40541 } } peers { permanent_uuid: "7e334a6d68eb4d98853ce401bd25949b" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 34229 } }
I20260806 04:57:27.799599 13071 raft_consensus.cc:374] T bfc13521019948d0b8af5deee7e0d0af P 4b26fa9b4a984025a159d0f38be81428 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 04:57:27.799695 13071 raft_consensus.cc:729] T bfc13521019948d0b8af5deee7e0d0af P 4b26fa9b4a984025a159d0f38be81428 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b26fa9b4a984025a159d0f38be81428, State: Initialized, Role: FOLLOWER
I20260806 04:57:27.799851 13073 raft_consensus.cc:348] T bfc13521019948d0b8af5deee7e0d0af P 61f45f84da5f4e89bd2a27ed0cdc051b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b26fa9b4a984025a159d0f38be81428" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 41697 } } peers { permanent_uuid: "61f45f84da5f4e89bd2a27ed0cdc051b" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 40541 } } peers { permanent_uuid: "7e334a6d68eb4d98853ce401bd25949b" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 34229 } }
I20260806 04:57:27.800084 13073 raft_consensus.cc:374] T bfc13521019948d0b8af5deee7e0d0af P 61f45f84da5f4e89bd2a27ed0cdc051b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 04:57:27.799986 13071 consensus_queue.cc:260] T bfc13521019948d0b8af5deee7e0d0af P 4b26fa9b4a984025a159d0f38be81428 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b26fa9b4a984025a159d0f38be81428" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 41697 } } peers { permanent_uuid: "61f45f84da5f4e89bd2a27ed0cdc051b" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 40541 } } peers { permanent_uuid: "7e334a6d68eb4d98853ce401bd25949b" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 34229 } }
I20260806 04:57:27.800170 13073 raft_consensus.cc:729] T bfc13521019948d0b8af5deee7e0d0af P 61f45f84da5f4e89bd2a27ed0cdc051b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 61f45f84da5f4e89bd2a27ed0cdc051b, State: Initialized, Role: FOLLOWER
I20260806 04:57:27.800222 13069 raft_consensus.cc:348] T bfc13521019948d0b8af5deee7e0d0af P 7e334a6d68eb4d98853ce401bd25949b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b26fa9b4a984025a159d0f38be81428" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 41697 } } peers { permanent_uuid: "61f45f84da5f4e89bd2a27ed0cdc051b" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 40541 } } peers { permanent_uuid: "7e334a6d68eb4d98853ce401bd25949b" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 34229 } }
I20260806 04:57:27.800462 13069 raft_consensus.cc:374] T bfc13521019948d0b8af5deee7e0d0af P 7e334a6d68eb4d98853ce401bd25949b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 04:57:27.800539 13069 raft_consensus.cc:729] T bfc13521019948d0b8af5deee7e0d0af P 7e334a6d68eb4d98853ce401bd25949b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e334a6d68eb4d98853ce401bd25949b, State: Initialized, Role: FOLLOWER
I20260806 04:57:27.800423 13073 consensus_queue.cc:260] T bfc13521019948d0b8af5deee7e0d0af P 61f45f84da5f4e89bd2a27ed0cdc051b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b26fa9b4a984025a159d0f38be81428" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 41697 } } peers { permanent_uuid: "61f45f84da5f4e89bd2a27ed0cdc051b" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 40541 } } peers { permanent_uuid: "7e334a6d68eb4d98853ce401bd25949b" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 34229 } }
I20260806 04:57:27.800652 13071 ts_tablet_manager.cc:1428] T bfc13521019948d0b8af5deee7e0d0af P 4b26fa9b4a984025a159d0f38be81428: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260806 04:57:27.801342 13069 consensus_queue.cc:260] T bfc13521019948d0b8af5deee7e0d0af P 7e334a6d68eb4d98853ce401bd25949b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b26fa9b4a984025a159d0f38be81428" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 41697 } } peers { permanent_uuid: "61f45f84da5f4e89bd2a27ed0cdc051b" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 40541 } } peers { permanent_uuid: "7e334a6d68eb4d98853ce401bd25949b" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 34229 } }
I20260806 04:57:27.802012 13069 ts_tablet_manager.cc:1428] T bfc13521019948d0b8af5deee7e0d0af P 7e334a6d68eb4d98853ce401bd25949b: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260806 04:57:27.803191 13073 ts_tablet_manager.cc:1428] T bfc13521019948d0b8af5deee7e0d0af P 61f45f84da5f4e89bd2a27ed0cdc051b: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260806 04:57:27.891454 13044 raft_consensus.cc:482] T bfc13521019948d0b8af5deee7e0d0af P 4b26fa9b4a984025a159d0f38be81428 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 04:57:27.891659 13044 raft_consensus.cc:504] T bfc13521019948d0b8af5deee7e0d0af P 4b26fa9b4a984025a159d0f38be81428 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b26fa9b4a984025a159d0f38be81428" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 41697 } } peers { permanent_uuid: "61f45f84da5f4e89bd2a27ed0cdc051b" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 40541 } } peers { permanent_uuid: "7e334a6d68eb4d98853ce401bd25949b" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 34229 } }
I20260806 04:57:27.892244 13044 leader_election.cc:288] T bfc13521019948d0b8af5deee7e0d0af P 4b26fa9b4a984025a159d0f38be81428 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 61f45f84da5f4e89bd2a27ed0cdc051b (127.9.96.66:40541), 7e334a6d68eb4d98853ce401bd25949b (127.9.96.67:34229)
I20260806 04:57:27.892906 12845 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bfc13521019948d0b8af5deee7e0d0af" candidate_uuid: "4b26fa9b4a984025a159d0f38be81428" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e334a6d68eb4d98853ce401bd25949b" is_pre_election: true
I20260806 04:57:27.892931 12639 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bfc13521019948d0b8af5deee7e0d0af" candidate_uuid: "4b26fa9b4a984025a159d0f38be81428" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "61f45f84da5f4e89bd2a27ed0cdc051b" is_pre_election: true
I20260806 04:57:27.893175 12845 raft_consensus.cc:2454] T bfc13521019948d0b8af5deee7e0d0af P 7e334a6d68eb4d98853ce401bd25949b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b26fa9b4a984025a159d0f38be81428 in term 0.
I20260806 04:57:27.893190 12639 raft_consensus.cc:2454] T bfc13521019948d0b8af5deee7e0d0af P 61f45f84da5f4e89bd2a27ed0cdc051b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b26fa9b4a984025a159d0f38be81428 in term 0.
I20260806 04:57:27.893721 12339 leader_election.cc:302] T bfc13521019948d0b8af5deee7e0d0af P 4b26fa9b4a984025a159d0f38be81428 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 4b26fa9b4a984025a159d0f38be81428, 61f45f84da5f4e89bd2a27ed0cdc051b, 7e334a6d68eb4d98853ce401bd25949b; no voters:
I20260806 04:57:27.894070 13044 raft_consensus.cc:2781] T bfc13521019948d0b8af5deee7e0d0af P 4b26fa9b4a984025a159d0f38be81428 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 04:57:27.894209 13044 raft_consensus.cc:482] T bfc13521019948d0b8af5deee7e0d0af P 4b26fa9b4a984025a159d0f38be81428 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 04:57:27.894282 13044 raft_consensus.cc:3037] T bfc13521019948d0b8af5deee7e0d0af P 4b26fa9b4a984025a159d0f38be81428 [term 0 FOLLOWER]: Advancing to term 1
I20260806 04:57:27.896339 13044 raft_consensus.cc:504] T bfc13521019948d0b8af5deee7e0d0af P 4b26fa9b4a984025a159d0f38be81428 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b26fa9b4a984025a159d0f38be81428" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 41697 } } peers { permanent_uuid: "61f45f84da5f4e89bd2a27ed0cdc051b" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 40541 } } peers { permanent_uuid: "7e334a6d68eb4d98853ce401bd25949b" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 34229 } }
I20260806 04:57:27.896881 13044 leader_election.cc:288] T bfc13521019948d0b8af5deee7e0d0af P 4b26fa9b4a984025a159d0f38be81428 [CANDIDATE]: Term 1 election: Requested vote from peers 61f45f84da5f4e89bd2a27ed0cdc051b (127.9.96.66:40541), 7e334a6d68eb4d98853ce401bd25949b (127.9.96.67:34229)
I20260806 04:57:27.897388 12639 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bfc13521019948d0b8af5deee7e0d0af" candidate_uuid: "4b26fa9b4a984025a159d0f38be81428" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "61f45f84da5f4e89bd2a27ed0cdc051b"
I20260806 04:57:27.897434 12845 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bfc13521019948d0b8af5deee7e0d0af" candidate_uuid: "4b26fa9b4a984025a159d0f38be81428" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e334a6d68eb4d98853ce401bd25949b"
I20260806 04:57:27.897600 12639 raft_consensus.cc:3037] T bfc13521019948d0b8af5deee7e0d0af P 61f45f84da5f4e89bd2a27ed0cdc051b [term 0 FOLLOWER]: Advancing to term 1
I20260806 04:57:27.897722 12845 raft_consensus.cc:3037] T bfc13521019948d0b8af5deee7e0d0af P 7e334a6d68eb4d98853ce401bd25949b [term 0 FOLLOWER]: Advancing to term 1
I20260806 04:57:27.899725 12639 raft_consensus.cc:2454] T bfc13521019948d0b8af5deee7e0d0af P 61f45f84da5f4e89bd2a27ed0cdc051b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b26fa9b4a984025a159d0f38be81428 in term 1.
I20260806 04:57:27.899844 12845 raft_consensus.cc:2454] T bfc13521019948d0b8af5deee7e0d0af P 7e334a6d68eb4d98853ce401bd25949b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b26fa9b4a984025a159d0f38be81428 in term 1.
I20260806 04:57:27.900195 12336 leader_election.cc:302] T bfc13521019948d0b8af5deee7e0d0af P 4b26fa9b4a984025a159d0f38be81428 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4b26fa9b4a984025a159d0f38be81428, 61f45f84da5f4e89bd2a27ed0cdc051b; no voters:
I20260806 04:57:27.900511 13044 raft_consensus.cc:2781] T bfc13521019948d0b8af5deee7e0d0af P 4b26fa9b4a984025a159d0f38be81428 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 04:57:27.900684 13044 raft_consensus.cc:686] T bfc13521019948d0b8af5deee7e0d0af P 4b26fa9b4a984025a159d0f38be81428 [term 1 LEADER]: Becoming Leader. State: Replica: 4b26fa9b4a984025a159d0f38be81428, State: Running, Role: LEADER
I20260806 04:57:27.900949 13044 consensus_queue.cc:237] T bfc13521019948d0b8af5deee7e0d0af P 4b26fa9b4a984025a159d0f38be81428 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b26fa9b4a984025a159d0f38be81428" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 41697 } } peers { permanent_uuid: "61f45f84da5f4e89bd2a27ed0cdc051b" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 40541 } } peers { permanent_uuid: "7e334a6d68eb4d98853ce401bd25949b" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 34229 } }
I20260806 04:57:27.903607 12210 catalog_manager.cc:5495] T bfc13521019948d0b8af5deee7e0d0af P 4b26fa9b4a984025a159d0f38be81428 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4b26fa9b4a984025a159d0f38be81428 (127.9.96.65). New cstate: current_term: 1 leader_uuid: "4b26fa9b4a984025a159d0f38be81428" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b26fa9b4a984025a159d0f38be81428" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 41697 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "61f45f84da5f4e89bd2a27ed0cdc051b" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 40541 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7e334a6d68eb4d98853ce401bd25949b" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 34229 } health_report { overall_health: UNKNOWN } } }
I20260806 04:57:27.931442 13100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:57:27.926219 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50054 (local address 127.9.96.65:41697)
0806 04:57:27.926805 (+ 586us) server_negotiation.cc:206] Beginning negotiation
0806 04:57:27.926809 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 04:57:27.926828 (+ 19us) server_negotiation.cc:407] Connection header received
0806 04:57:27.926877 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:57:27.926880 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:57:27.926937 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:57:27.927019 (+ 82us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 04:57:27.927432 (+ 413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:57:27.928246 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:57:27.929574 (+ 1328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:57:27.930089 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:57:27.930205 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:57:27.930690 (+ 485us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 04:57:27.930795 (+ 105us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 04:57:27.930867 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0806 04:57:27.930992 (+ 125us) server_negotiation.cc:299] Negotiation successful
0806 04:57:27.931071 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":485,"thread_start_us":340,"threads_started":1}
I20260806 04:57:27.940409 12845 raft_consensus.cc:1261] T bfc13521019948d0b8af5deee7e0d0af P 7e334a6d68eb4d98853ce401bd25949b [term 1 FOLLOWER]: Refusing update from remote peer 4b26fa9b4a984025a159d0f38be81428: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 04:57:27.940557 12639 raft_consensus.cc:1261] T bfc13521019948d0b8af5deee7e0d0af P 61f45f84da5f4e89bd2a27ed0cdc051b [term 1 FOLLOWER]: Refusing update from remote peer 4b26fa9b4a984025a159d0f38be81428: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 04:57:27.940925 13102 consensus_queue.cc:1035] T bfc13521019948d0b8af5deee7e0d0af P 4b26fa9b4a984025a159d0f38be81428 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e334a6d68eb4d98853ce401bd25949b" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 34229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 04:57:27.941406 13028 consensus_queue.cc:1035] T bfc13521019948d0b8af5deee7e0d0af P 4b26fa9b4a984025a159d0f38be81428 [LEADER]: Connected to new peer: Peer: permanent_uuid: "61f45f84da5f4e89bd2a27ed0cdc051b" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 40541 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260806 04:57:27.962289 13025 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260806 04:57:29.533927 12210 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47870:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 04:57:29.534232 12210 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47870:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 04:57:29.538092 12210 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ebfef4bb062640d9bf626f0e62f6b98c: Sending DeleteTablet for 3 replicas of tablet bfc13521019948d0b8af5deee7e0d0af
I20260806 04:57:29.538794 12405 tablet_service.cc:1514] Processing DeleteTablet for tablet bfc13521019948d0b8af5deee7e0d0af with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 04:57:29 UTC) from {username='kudu', principal='kudu/127.9.96.126@KRBTEST.COM'} at 127.0.0.1:50012
I20260806 04:57:29.538987 12608 tablet_service.cc:1514] Processing DeleteTablet for tablet bfc13521019948d0b8af5deee7e0d0af with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 04:57:29 UTC) from {username='kudu', principal='kudu/127.9.96.126@KRBTEST.COM'} at 127.0.0.1:44726
I20260806 04:57:29.538988 12806 tablet_service.cc:1514] Processing DeleteTablet for tablet bfc13521019948d0b8af5deee7e0d0af with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 04:57:29 UTC) from {username='kudu', principal='kudu/127.9.96.126@KRBTEST.COM'} at 127.0.0.1:60914
I20260806 04:57:29.539677 9601 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu pid 12306
I20260806 04:57:29.545352 13128 tablet_replica.cc:330] T bfc13521019948d0b8af5deee7e0d0af P 4b26fa9b4a984025a159d0f38be81428: stopping tablet replica
I20260806 04:57:29.545634 13128 raft_consensus.cc:2229] T bfc13521019948d0b8af5deee7e0d0af P 4b26fa9b4a984025a159d0f38be81428 [term 1 LEADER]: Raft consensus shutting down.
I20260806 04:57:29.547227 13128 raft_consensus.cc:2258] T bfc13521019948d0b8af5deee7e0d0af P 4b26fa9b4a984025a159d0f38be81428 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 04:57:29.550637 13128 ts_tablet_manager.cc:1905] T bfc13521019948d0b8af5deee7e0d0af P 4b26fa9b4a984025a159d0f38be81428: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 04:57:29.553701 13130 tablet_replica.cc:330] T bfc13521019948d0b8af5deee7e0d0af P 7e334a6d68eb4d98853ce401bd25949b: stopping tablet replica
I20260806 04:57:29.553867 13129 tablet_replica.cc:330] T bfc13521019948d0b8af5deee7e0d0af P 61f45f84da5f4e89bd2a27ed0cdc051b: stopping tablet replica
I20260806 04:57:29.553901 13130 raft_consensus.cc:2229] T bfc13521019948d0b8af5deee7e0d0af P 7e334a6d68eb4d98853ce401bd25949b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 04:57:29.554080 13129 raft_consensus.cc:2229] T bfc13521019948d0b8af5deee7e0d0af P 61f45f84da5f4e89bd2a27ed0cdc051b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 04:57:29.554155 13130 raft_consensus.cc:2258] T bfc13521019948d0b8af5deee7e0d0af P 7e334a6d68eb4d98853ce401bd25949b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 04:57:29.554384 13129 raft_consensus.cc:2258] T bfc13521019948d0b8af5deee7e0d0af P 61f45f84da5f4e89bd2a27ed0cdc051b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 04:57:29.555455 13129 ts_tablet_manager.cc:1905] T bfc13521019948d0b8af5deee7e0d0af P 61f45f84da5f4e89bd2a27ed0cdc051b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 04:57:29.556092 13130 ts_tablet_manager.cc:1905] T bfc13521019948d0b8af5deee7e0d0af P 7e334a6d68eb4d98853ce401bd25949b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 04:57:29.558913 13128 ts_tablet_manager.cc:1918] T bfc13521019948d0b8af5deee7e0d0af P 4b26fa9b4a984025a159d0f38be81428: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260806 04:57:29.559104 13128 log.cc:1196] T bfc13521019948d0b8af5deee7e0d0af P 4b26fa9b4a984025a159d0f38be81428: Deleting WAL directory at /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785992227050689-9601-0/minicluster-data/ts-0/wal/wals/bfc13521019948d0b8af5deee7e0d0af
Aug 06 04:57:29 dist-test-slave-x4qh krb5kdc[12131](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992247, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.9.96.65@KRBTEST.COM
I20260806 04:57:29.559682 13128 ts_tablet_manager.cc:1939] T bfc13521019948d0b8af5deee7e0d0af P 4b26fa9b4a984025a159d0f38be81428: Deleting consensus metadata
I20260806 04:57:29.560928 12187 catalog_manager.cc:4841] TS 4b26fa9b4a984025a159d0f38be81428 (127.9.96.65:41697): tablet bfc13521019948d0b8af5deee7e0d0af (table test-table [id=ac7824ff882d462d94377ab3bca6cbfc]) successfully deleted
I20260806 04:57:29.561944 13129 ts_tablet_manager.cc:1918] T bfc13521019948d0b8af5deee7e0d0af P 61f45f84da5f4e89bd2a27ed0cdc051b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260806 04:57:29.562067 13129 log.cc:1196] T bfc13521019948d0b8af5deee7e0d0af P 61f45f84da5f4e89bd2a27ed0cdc051b: Deleting WAL directory at /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785992227050689-9601-0/minicluster-data/ts-1/wal/wals/bfc13521019948d0b8af5deee7e0d0af
I20260806 04:57:29.562537 13129 ts_tablet_manager.cc:1939] T bfc13521019948d0b8af5deee7e0d0af P 61f45f84da5f4e89bd2a27ed0cdc051b: Deleting consensus metadata
I20260806 04:57:29.562592 13130 ts_tablet_manager.cc:1918] T bfc13521019948d0b8af5deee7e0d0af P 7e334a6d68eb4d98853ce401bd25949b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260806 04:57:29.562738 13130 log.cc:1196] T bfc13521019948d0b8af5deee7e0d0af P 7e334a6d68eb4d98853ce401bd25949b: Deleting WAL directory at /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785992227050689-9601-0/minicluster-data/ts-2/wal/wals/bfc13521019948d0b8af5deee7e0d0af
I20260806 04:57:29.563207 13130 ts_tablet_manager.cc:1939] T bfc13521019948d0b8af5deee7e0d0af P 7e334a6d68eb4d98853ce401bd25949b: Deleting consensus metadata
I20260806 04:57:29.563638 12185 catalog_manager.cc:4841] TS 61f45f84da5f4e89bd2a27ed0cdc051b (127.9.96.66:40541): tablet bfc13521019948d0b8af5deee7e0d0af (table test-table [id=ac7824ff882d462d94377ab3bca6cbfc]) successfully deleted
I20260806 04:57:29.564267 12187 catalog_manager.cc:4841] TS 7e334a6d68eb4d98853ce401bd25949b (127.9.96.67:34229): tablet bfc13521019948d0b8af5deee7e0d0af (table test-table [id=ac7824ff882d462d94377ab3bca6cbfc]) successfully deleted
I20260806 04:57:29.567437 13132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:57:29.541359 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50064 (local address 127.9.96.65:41697)
0806 04:57:29.542701 (+ 1342us) server_negotiation.cc:206] Beginning negotiation
0806 04:57:29.542710 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 04:57:29.543436 (+ 726us) server_negotiation.cc:407] Connection header received
0806 04:57:29.543596 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:57:29.543604 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:57:29.543727 (+ 123us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:57:29.543932 (+ 205us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:57:29.547939 (+ 4007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:57:29.549401 (+ 1462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:57:29.554429 (+ 5028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:57:29.555149 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:57:29.555252 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:57:29.560791 (+ 5539us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:57:29.560811 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:57:29.560845 (+ 34us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:57:29.560895 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:57:29.564552 (+ 3657us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:57:29.565178 (+ 626us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:57:29.565188 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:57:29.565191 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:57:29.565350 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:57:29.565744 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:57:29.565749 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:57:29.565753 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:57:29.566166 (+ 413us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:57:29.566464 (+ 298us) server_negotiation.cc:1035] Waiting for connection context
0806 04:57:29.566783 (+ 319us) server_negotiation.cc:299] Negotiation successful
0806 04:57:29.567027 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1142,"thread_start_us":930,"threads_started":1}
W20260806 04:57:29.568125 12499 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50064
W20260806 04:57:29.568730 9601 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu pid 12306: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260806 04:57:29.568914 9601 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu with pid 12306
I20260806 04:57:29.604454 9601 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu pid 12512
Aug 06 04:57:29 dist-test-slave-x4qh krb5kdc[12131](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992247, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.9.96.66@KRBTEST.COM
I20260806 04:57:29.620131 13133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:57:29.605224 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44754 (local address 127.9.96.66:40541)
0806 04:57:29.605716 (+ 492us) server_negotiation.cc:206] Beginning negotiation
0806 04:57:29.605721 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 04:57:29.605738 (+ 17us) server_negotiation.cc:407] Connection header received
0806 04:57:29.605792 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:57:29.605796 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:57:29.605864 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:57:29.605963 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:57:29.607314 (+ 1351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:57:29.608170 (+ 856us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:57:29.609398 (+ 1228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:57:29.609832 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:57:29.609928 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:57:29.614267 (+ 4339us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:57:29.614280 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:57:29.614295 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:57:29.614327 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:57:29.617396 (+ 3069us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:57:29.618096 (+ 700us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:57:29.618103 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:57:29.618105 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:57:29.618236 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:57:29.618564 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:57:29.618567 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:57:29.618569 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:57:29.619002 (+ 433us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:57:29.619221 (+ 219us) server_negotiation.cc:1035] Waiting for connection context
0806 04:57:29.619519 (+ 298us) server_negotiation.cc:299] Negotiation successful
0806 04:57:29.619783 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":373,"thread_start_us":249,"threads_started":1}
W20260806 04:57:29.620781 12706 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:44754
W20260806 04:57:29.621428 9601 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu pid 12512: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260806 04:57:29.621546 9601 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu with pid 12512
I20260806 04:57:29.649907 9601 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu pid 12720
Aug 06 04:57:29 dist-test-slave-x4qh krb5kdc[12131](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992247, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.9.96.67@KRBTEST.COM
I20260806 04:57:29.664983 13134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:57:29.650703 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60922 (local address 127.9.96.67:34229)
0806 04:57:29.651338 (+ 635us) server_negotiation.cc:206] Beginning negotiation
0806 04:57:29.651343 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 04:57:29.651359 (+ 16us) server_negotiation.cc:407] Connection header received
0806 04:57:29.651477 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:57:29.651481 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:57:29.651540 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:57:29.651628 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:57:29.653014 (+ 1386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:57:29.653758 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:57:29.654892 (+ 1134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:57:29.655308 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:57:29.655567 (+ 259us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:57:29.659821 (+ 4254us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:57:29.659835 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:57:29.659851 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:57:29.659887 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:57:29.662567 (+ 2680us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:57:29.663114 (+ 547us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:57:29.663120 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:57:29.663122 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:57:29.663264 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:57:29.663633 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:57:29.663636 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:57:29.663638 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:57:29.664006 (+ 368us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:57:29.664216 (+ 210us) server_negotiation.cc:1035] Waiting for connection context
0806 04:57:29.664523 (+ 307us) server_negotiation.cc:299] Negotiation successful
0806 04:57:29.664675 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":538,"thread_start_us":349,"threads_started":1}
W20260806 04:57:29.665875 12899 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:60922
W20260806 04:57:29.666519 9601 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu pid 12720: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260806 04:57:29.666692 9601 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu with pid 12720
I20260806 04:57:29.692076 9601 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu pid 12159
I20260806 04:57:29.693261 12261 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 04:57:29.808056 9601 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu with pid 12159
2026-08-06T04:57:29Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (8694 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785992227050689-9601-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 04:57:29 dist-test-slave-x4qh krb5kdc[13138](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 04:57:29 dist-test-slave-x4qh krb5kdc[13138](info): set up 2 sockets
Aug 06 04:57:29 dist-test-slave-x4qh krb5kdc[13138](info): commencing operation
krb5kdc: starting...
W20260806 04:57:31.093593 9601 mini_kdc.cc:121] Time spent starting KDC: real 1.232s user 0.002s sys 0.018s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785992227050689-9601-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785992227050689-9601-0/krb5kdc/test-user.keytab.
Aug 06 04:57:31 dist-test-slave-x4qh krb5kdc[13138](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992251, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T04:57:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T04:57:31Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.96.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.96.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.96.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.126.keytab.
Entry for principal kudu/127.9.96.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.126.keytab.
I20260806 04:57:31.289009 9601 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
/tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785992227050689-9601-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785992227050689-9601-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785992227050689-9601-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785992227050689-9601-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.96.126:35333
--webserver_interface=127.9.96.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.126.keytab
--principal=kudu/127.9.96.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.96.84:37793
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.96.126:35333
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785992227050689-9601-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785992227050689-9601-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785992227050689-9601-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 04:57:31.614845 13159 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 04:57:31.615224 13159 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 04:57:31.615425 13159 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 04:57:31.622416 13159 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 04:57:31.622574 13159 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 04:57:31.622675 13159 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 04:57:31.622781 13159 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 04:57:31.622859 13159 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 04:57:31.632918 13159 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.96.84:37793
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785992227050689-9601-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785992227050689-9601-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.96.126:35333
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.126.keytab
--principal=kudu/127.9.96.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.96.126:35333
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785992227050689-9601-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.96.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785992227050689-9601-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 03:56:12 UTC on ccycloud.quasar-guijbm.root.comops.site
build id 4569618
ASAN enabled
I20260806 04:57:31.634188 13159 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 04:57:31.635579 13159 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 04:57:31.647570 13166 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:57:31.647596 13167 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:57:31.647677 13170 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:57:31.648396 13159 server_base.cc:1034] running on GCE node
I20260806 04:57:31.649159 13159 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 04:57:31.650444 13159 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 04:57:31.651669 13159 hybrid_clock.cc:648] HybridClock initialized: now 1785992251651621 us; error 59 us; skew 500 ppm
I20260806 04:57:31.652014 13159 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 04:57:31 dist-test-slave-x4qh krb5kdc[13138](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992251, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.96.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 04:57:31.655104 13159 init.cc:377] Logged in from keytab as kudu/127.9.96.126@KRBTEST.COM (short username kudu)
I20260806 04:57:31.656662 13159 webserver.cc:460] Webserver started at http://127.9.96.126:39487/ using document root <none> and password file <none>
I20260806 04:57:31.657295 13159 fs_manager.cc:359] Metadata directory not provided
I20260806 04:57:31.657415 13159 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 04:57:31.657658 13159 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 04:57:31.660609 13159 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785992227050689-9601-0/minicluster-data/master-0/data/instance:
uuid: "0a2188932f9849fa9cd5ae634684241d"
format_stamp: "Formatted at 2026-08-06 04:57:31 on dist-test-slave-x4qh"
server_key: "caaf7ff6f9ab41a6a1584ec5f6c6b1f4"
server_key_iv: "f1b2c68490d18314f41c0672ade880d4"
server_key_version: "encryptionkey@0"
I20260806 04:57:31.661410 13159 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785992227050689-9601-0/minicluster-data/master-0/wal/instance:
uuid: "0a2188932f9849fa9cd5ae634684241d"
format_stamp: "Formatted at 2026-08-06 04:57:31 on dist-test-slave-x4qh"
server_key: "caaf7ff6f9ab41a6a1584ec5f6c6b1f4"
server_key_iv: "f1b2c68490d18314f41c0672ade880d4"
server_key_version: "encryptionkey@0"
I20260806 04:57:31.666453 13159 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260806 04:57:31.669723 13178 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 04:57:31.670935 13159 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260806 04:57:31.671116 13159 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785992227050689-9601-0/minicluster-data/master-0/data,/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785992227050689-9601-0/minicluster-data/master-0/wal
uuid: "0a2188932f9849fa9cd5ae634684241d"
format_stamp: "Formatted at 2026-08-06 04:57:31 on dist-test-slave-x4qh"
server_key: "caaf7ff6f9ab41a6a1584ec5f6c6b1f4"
server_key_iv: "f1b2c68490d18314f41c0672ade880d4"
server_key_version: "encryptionkey@0"
I20260806 04:57:31.671295 13159 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785992227050689-9601-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785992227050689-9601-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785992227050689-9601-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 04:57:31.693799 13159 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 04:57:31.697635 13159 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 04:57:31.697935 13159 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 04:57:31.716650 13159 rpc_server.cc:307] RPC server started. Bound to: 127.9.96.126:35333
I20260806 04:57:31.716696 13253 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.96.126:35333 every 8 connection(s)
I20260806 04:57:31.717957 13159 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785992227050689-9601-0/minicluster-data/master-0/data/info.pb
I20260806 04:57:31.721487 13257 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 04:57:31.727006 9601 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu as pid 13159
I20260806 04:57:31.727258 9601 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785992227050689-9601-0/minicluster-data/master-0/wal/instance
I20260806 04:57:31.727924 9601 external_mini_cluster.cc:1428] Setting key e08555dcd3816b8c8b7264efdcec9bde
I20260806 04:57:31.731925 13257 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0a2188932f9849fa9cd5ae634684241d: Bootstrap starting.
I20260806 04:57:31.735610 13257 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0a2188932f9849fa9cd5ae634684241d: Neither blocks nor log segments found. Creating new log.
I20260806 04:57:31.736627 13257 log.cc:824] T 00000000000000000000000000000000 P 0a2188932f9849fa9cd5ae634684241d: Log is configured to *not* fsync() on all Append() calls
I20260806 04:57:31.739605 13257 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0a2188932f9849fa9cd5ae634684241d: No bootstrap required, opened a new log
Aug 06 04:57:31 dist-test-slave-x4qh krb5kdc[13138](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992251, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.96.126@KRBTEST.COM
I20260806 04:57:31.747839 13257 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0a2188932f9849fa9cd5ae634684241d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a2188932f9849fa9cd5ae634684241d" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 35333 } }
I20260806 04:57:31.748157 13257 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0a2188932f9849fa9cd5ae634684241d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 04:57:31.748279 13257 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0a2188932f9849fa9cd5ae634684241d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a2188932f9849fa9cd5ae634684241d, State: Initialized, Role: FOLLOWER
I20260806 04:57:31.748962 13257 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0a2188932f9849fa9cd5ae634684241d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a2188932f9849fa9cd5ae634684241d" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 35333 } }
I20260806 04:57:31.749267 13257 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0a2188932f9849fa9cd5ae634684241d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 04:57:31.749456 13257 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0a2188932f9849fa9cd5ae634684241d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 04:57:31.749650 13257 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0a2188932f9849fa9cd5ae634684241d [term 0 FOLLOWER]: Advancing to term 1
I20260806 04:57:31.751300 13261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:57:31.729867 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43564 (local address 127.9.96.126:35333)
0806 04:57:31.730653 (+ 786us) server_negotiation.cc:206] Beginning negotiation
0806 04:57:31.730663 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 04:57:31.730700 (+ 37us) server_negotiation.cc:407] Connection header received
0806 04:57:31.732431 (+ 1731us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:57:31.732452 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:57:31.732781 (+ 329us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:57:31.733078 (+ 297us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:57:31.735100 (+ 2022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:57:31.736516 (+ 1416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:57:31.738193 (+ 1677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:57:31.738813 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:57:31.738953 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:57:31.743657 (+ 4704us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:57:31.743665 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:57:31.743678 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:57:31.743717 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:57:31.746925 (+ 3208us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:57:31.747683 (+ 758us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:57:31.747698 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:57:31.747704 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:57:31.747874 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:57:31.748311 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:57:31.748317 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:57:31.748322 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:57:31.748927 (+ 605us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:57:31.749240 (+ 313us) server_negotiation.cc:1035] Waiting for connection context
0806 04:57:31.749607 (+ 367us) server_negotiation.cc:299] Negotiation successful
0806 04:57:31.750031 (+ 424us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":601,"thread_start_us":416,"threads_started":1}
I20260806 04:57:31.752892 13257 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0a2188932f9849fa9cd5ae634684241d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a2188932f9849fa9cd5ae634684241d" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 35333 } }
I20260806 04:57:31.753399 13257 leader_election.cc:302] T 00000000000000000000000000000000 P 0a2188932f9849fa9cd5ae634684241d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0a2188932f9849fa9cd5ae634684241d; no voters:
I20260806 04:57:31.754133 13257 leader_election.cc:288] T 00000000000000000000000000000000 P 0a2188932f9849fa9cd5ae634684241d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 04:57:31.754294 13263 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0a2188932f9849fa9cd5ae634684241d [term 1 FOLLOWER]: Leader election won for term 1
I20260806 04:57:31.755036 13263 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0a2188932f9849fa9cd5ae634684241d [term 1 LEADER]: Becoming Leader. State: Replica: 0a2188932f9849fa9cd5ae634684241d, State: Running, Role: LEADER
I20260806 04:57:31.755589 13263 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0a2188932f9849fa9cd5ae634684241d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a2188932f9849fa9cd5ae634684241d" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 35333 } }
I20260806 04:57:31.756114 13257 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0a2188932f9849fa9cd5ae634684241d [sys.catalog]: configured and running, proceeding with master startup.
I20260806 04:57:31.758371 13264 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0a2188932f9849fa9cd5ae634684241d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0a2188932f9849fa9cd5ae634684241d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a2188932f9849fa9cd5ae634684241d" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 35333 } } }
I20260806 04:57:31.758430 13266 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0a2188932f9849fa9cd5ae634684241d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0a2188932f9849fa9cd5ae634684241d. Latest consensus state: current_term: 1 leader_uuid: "0a2188932f9849fa9cd5ae634684241d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a2188932f9849fa9cd5ae634684241d" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 35333 } } }
I20260806 04:57:31.758772 13264 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0a2188932f9849fa9cd5ae634684241d [sys.catalog]: This master's current role is: LEADER
I20260806 04:57:31.758845 13266 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0a2188932f9849fa9cd5ae634684241d [sys.catalog]: This master's current role is: LEADER
I20260806 04:57:31.768699 13270 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 04:57:31.774722 13270 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 04:57:31.783948 13270 catalog_manager.cc:1347] Generated new cluster ID: 1b423987da424d008d5d433496b67965
I20260806 04:57:31.784062 13270 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 04:57:31.803392 13270 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 04:57:31.805285 13270 catalog_manager.cc:1504] Loading token signing keys...
I20260806 04:57:31.812814 13270 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0a2188932f9849fa9cd5ae634684241d: Generated new TSK 0
I20260806 04:57:31.813879 13270 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.9.96.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.96.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.96.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.65.keytab.
Entry for principal kudu/127.9.96.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.65.keytab.
I20260806 04:57:31.864781 9601 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
/tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785992227050689-9601-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785992227050689-9601-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785992227050689-9601-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785992227050689-9601-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.96.65:0
--local_ip_for_outbound_sockets=127.9.96.65
--webserver_interface=127.9.96.65
--webserver_port=0
--tserver_master_addrs=127.9.96.126:35333
--keytab_file=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.65.keytab
--principal=kudu/127.9.96.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.96.84:37793
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785992227050689-9601-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785992227050689-9601-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785992227050689-9601-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 04:57:32.191824 13295 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 04:57:32.192137 13295 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 04:57:32.192319 13295 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 04:57:32.199733 13295 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 04:57:32.199915 13295 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 04:57:32.200086 13295 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.96.65
I20260806 04:57:32.209754 13295 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.96.84:37793
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785992227050689-9601-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785992227050689-9601-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.65.keytab
--principal=kudu/127.9.96.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.96.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785992227050689-9601-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.96.65
--webserver_port=0
--tserver_master_addrs=127.9.96.126:35333
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.96.65
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785992227050689-9601-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 03:56:12 UTC on ccycloud.quasar-guijbm.root.comops.site
build id 4569618
ASAN enabled
I20260806 04:57:32.211023 13295 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 04:57:32.212396 13295 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 04:57:32.225060 13301 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:57:32.225266 13302 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:57:32.225270 13307 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:57:32.226279 13295 server_base.cc:1034] running on GCE node
I20260806 04:57:32.226936 13295 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 04:57:32.227878 13295 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 04:57:32.229105 13295 hybrid_clock.cc:648] HybridClock initialized: now 1785992252229075 us; error 40 us; skew 500 ppm
I20260806 04:57:32.229454 13295 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 04:57:32 dist-test-slave-x4qh krb5kdc[13138](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992252, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.96.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 04:57:32.232570 13295 init.cc:377] Logged in from keytab as kudu/127.9.96.65@KRBTEST.COM (short username kudu)
I20260806 04:57:32.233940 13295 webserver.cc:460] Webserver started at http://127.9.96.65:34623/ using document root <none> and password file <none>
I20260806 04:57:32.234642 13295 fs_manager.cc:359] Metadata directory not provided
I20260806 04:57:32.234788 13295 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 04:57:32.235100 13295 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 04:57:32.238078 13295 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785992227050689-9601-0/minicluster-data/ts-0/data/instance:
uuid: "3daed170e65745d98725e4d89b1713c9"
format_stamp: "Formatted at 2026-08-06 04:57:32 on dist-test-slave-x4qh"
server_key: "1a4dd8c0af557145b8c90765f6412d26"
server_key_iv: "d63e39d1e63e550d1146964e038a2c7f"
server_key_version: "encryptionkey@0"
I20260806 04:57:32.238885 13295 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785992227050689-9601-0/minicluster-data/ts-0/wal/instance:
uuid: "3daed170e65745d98725e4d89b1713c9"
format_stamp: "Formatted at 2026-08-06 04:57:32 on dist-test-slave-x4qh"
server_key: "1a4dd8c0af557145b8c90765f6412d26"
server_key_iv: "d63e39d1e63e550d1146964e038a2c7f"
server_key_version: "encryptionkey@0"
I20260806 04:57:32.245378 13295 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.005s
I20260806 04:57:32.248759 13318 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 04:57:32.250017 13295 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 04:57:32.250195 13295 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785992227050689-9601-0/minicluster-data/ts-0/data,/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785992227050689-9601-0/minicluster-data/ts-0/wal
uuid: "3daed170e65745d98725e4d89b1713c9"
format_stamp: "Formatted at 2026-08-06 04:57:32 on dist-test-slave-x4qh"
server_key: "1a4dd8c0af557145b8c90765f6412d26"
server_key_iv: "d63e39d1e63e550d1146964e038a2c7f"
server_key_version: "encryptionkey@0"
I20260806 04:57:32.250399 13295 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785992227050689-9601-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785992227050689-9601-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785992227050689-9601-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 04:57:32.266956 13295 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 04:57:32.270659 13295 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 04:57:32.270970 13295 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 04:57:32.272113 13295 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 04:57:32.273605 13295 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 04:57:32.273713 13295 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 04:57:32.273877 13295 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 04:57:32.273953 13295 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 04:57:32.308660 13295 rpc_server.cc:307] RPC server started. Bound to: 127.9.96.65:44221
I20260806 04:57:32.308756 13493 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.96.65:44221 every 8 connection(s)
I20260806 04:57:32.309988 13295 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785992227050689-9601-0/minicluster-data/ts-0/data/info.pb
I20260806 04:57:32.316121 9601 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu as pid 13295
I20260806 04:57:32.316375 9601 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785992227050689-9601-0/minicluster-data/ts-0/wal/instance
I20260806 04:57:32.317040 9601 external_mini_cluster.cc:1428] Setting key 3067f2ea857f5b6f92e32d4fdc6b070c
Aug 06 04:57:32 dist-test-slave-x4qh krb5kdc[13138](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992252, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.96.65@KRBTEST.COM for kudu/127.9.96.126@KRBTEST.COM
I20260806 04:57:32.332624 13496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:57:32.313091 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.96.65:58611 (local address 127.9.96.126:35333)
0806 04:57:32.313546 (+ 455us) server_negotiation.cc:206] Beginning negotiation
0806 04:57:32.313551 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 04:57:32.314418 (+ 867us) server_negotiation.cc:407] Connection header received
0806 04:57:32.316347 (+ 1929us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:57:32.316353 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:57:32.316454 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:57:32.316608 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:57:32.318327 (+ 1719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:57:32.319230 (+ 903us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:57:32.320842 (+ 1612us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:57:32.321243 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:57:32.321376 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:57:32.325487 (+ 4111us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:57:32.325492 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:57:32.325495 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:57:32.325528 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:57:32.329039 (+ 3511us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:57:32.329885 (+ 846us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:57:32.329894 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:57:32.329899 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:57:32.330084 (+ 185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:57:32.330541 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:57:32.330548 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:57:32.330554 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:57:32.330963 (+ 409us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:57:32.331116 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0806 04:57:32.331874 (+ 758us) server_negotiation.cc:299] Negotiation successful
0806 04:57:32.332153 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":341,"thread_start_us":251,"threads_started":1}
I20260806 04:57:32.333241 13497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:57:32.313536 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:35333 (local address 127.9.96.65:58611)
0806 04:57:32.314252 (+ 716us) negotiation.cc:107] Waiting for socket to connect
0806 04:57:32.314297 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0806 04:57:32.316078 (+ 1781us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:57:32.316848 (+ 770us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:57:32.316857 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:57:32.317448 (+ 591us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:57:32.318160 (+ 712us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:57:32.318181 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:57:32.319440 (+ 1259us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:57:32.319444 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:57:32.320684 (+ 1240us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:57:32.320696 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:57:32.321490 (+ 794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:57:32.321498 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:57:32.321719 (+ 221us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:57:32.322461 (+ 742us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:57:32.322488 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:57:32.325267 (+ 2779us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 04:57:32.329275 (+ 4008us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:57:32.329282 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:57:32.329293 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:57:32.329709 (+ 416us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:57:32.330223 (+ 514us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:57:32.330226 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:57:32.330228 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:57:32.330419 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:57:32.331094 (+ 675us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 04:57:32.331100 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 04:57:32.331462 (+ 362us) client_negotiation.cc:769] Sending connection context
0806 04:57:32.331709 (+ 247us) client_negotiation.cc:240] Negotiation successful
0806 04:57:32.332084 (+ 375us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":486,"thread_start_us":291,"threads_started":1}
I20260806 04:57:32.334805 13494 heartbeater.cc:347] Connected to a master server at 127.9.96.126:35333
I20260806 04:57:32.335163 13494 heartbeater.cc:464] Registering TS with master...
I20260806 04:57:32.335953 13494 heartbeater.cc:511] Master 127.9.96.126:35333 requested a full tablet report, sending...
I20260806 04:57:32.338085 13208 ts_manager.cc:194] Registered new tserver with Master: 3daed170e65745d98725e4d89b1713c9 (127.9.96.65:44221)
WARNING: no policy specified for kudu/127.9.96.66@KRBTEST.COM; defaulting to no policy
I20260806 04:57:32.339740 13208 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.96.65@KRBTEST.COM'} at 127.9.96.65:58611
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.96.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.96.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.66.keytab.
Entry for principal kudu/127.9.96.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.66.keytab.
I20260806 04:57:32.364966 9601 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
/tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785992227050689-9601-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785992227050689-9601-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785992227050689-9601-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785992227050689-9601-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.96.66:0
--local_ip_for_outbound_sockets=127.9.96.66
--webserver_interface=127.9.96.66
--webserver_port=0
--tserver_master_addrs=127.9.96.126:35333
--keytab_file=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.66.keytab
--principal=kudu/127.9.96.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.96.84:37793
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785992227050689-9601-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785992227050689-9601-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785992227050689-9601-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 04:57:32.690320 13500 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 04:57:32.690629 13500 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 04:57:32.690845 13500 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 04:57:32.698021 13500 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 04:57:32.698155 13500 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 04:57:32.698371 13500 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.96.66
I20260806 04:57:32.708443 13500 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.96.84:37793
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785992227050689-9601-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785992227050689-9601-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.66.keytab
--principal=kudu/127.9.96.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.96.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785992227050689-9601-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.96.66
--webserver_port=0
--tserver_master_addrs=127.9.96.126:35333
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.96.66
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785992227050689-9601-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 03:56:12 UTC on ccycloud.quasar-guijbm.root.comops.site
build id 4569618
ASAN enabled
I20260806 04:57:32.709622 13500 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 04:57:32.710825 13500 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 04:57:32.727255 13509 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:57:32.727491 13511 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:57:32.727286 13508 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:57:32.728659 13500 server_base.cc:1034] running on GCE node
I20260806 04:57:32.729280 13500 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 04:57:32.730131 13500 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 04:57:32.731355 13500 hybrid_clock.cc:648] HybridClock initialized: now 1785992252731310 us; error 38 us; skew 500 ppm
I20260806 04:57:32.731700 13500 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 04:57:32 dist-test-slave-x4qh krb5kdc[13138](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992252, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.96.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 04:57:32.737764 13500 init.cc:377] Logged in from keytab as kudu/127.9.96.66@KRBTEST.COM (short username kudu)
I20260806 04:57:32.739238 13500 webserver.cc:460] Webserver started at http://127.9.96.66:43911/ using document root <none> and password file <none>
I20260806 04:57:32.739961 13500 fs_manager.cc:359] Metadata directory not provided
I20260806 04:57:32.740108 13500 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 04:57:32.740427 13500 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 04:57:32.743283 13500 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785992227050689-9601-0/minicluster-data/ts-1/data/instance:
uuid: "7512054f59a34dc19913a5deb9b1b8b3"
format_stamp: "Formatted at 2026-08-06 04:57:32 on dist-test-slave-x4qh"
server_key: "4c5fcd3ce53ad9cde797abcecdbb183c"
server_key_iv: "f9079e275333418b37dd52e71eac4d7b"
server_key_version: "encryptionkey@0"
I20260806 04:57:32.744180 13500 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785992227050689-9601-0/minicluster-data/ts-1/wal/instance:
uuid: "7512054f59a34dc19913a5deb9b1b8b3"
format_stamp: "Formatted at 2026-08-06 04:57:32 on dist-test-slave-x4qh"
server_key: "4c5fcd3ce53ad9cde797abcecdbb183c"
server_key_iv: "f9079e275333418b37dd52e71eac4d7b"
server_key_version: "encryptionkey@0"
I20260806 04:57:32.749346 13500 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260806 04:57:32.752874 13526 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 04:57:32.754454 13500 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 04:57:32.754630 13500 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785992227050689-9601-0/minicluster-data/ts-1/data,/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785992227050689-9601-0/minicluster-data/ts-1/wal
uuid: "7512054f59a34dc19913a5deb9b1b8b3"
format_stamp: "Formatted at 2026-08-06 04:57:32 on dist-test-slave-x4qh"
server_key: "4c5fcd3ce53ad9cde797abcecdbb183c"
server_key_iv: "f9079e275333418b37dd52e71eac4d7b"
server_key_version: "encryptionkey@0"
I20260806 04:57:32.754915 13500 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785992227050689-9601-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785992227050689-9601-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785992227050689-9601-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 04:57:32.771721 13500 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 04:57:32.776180 13500 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 04:57:32.776467 13500 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 04:57:32.777420 13500 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 04:57:32.778743 13500 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 04:57:32.778853 13500 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 04:57:32.778996 13500 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 04:57:32.779064 13500 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 04:57:32.814006 13500 rpc_server.cc:307] RPC server started. Bound to: 127.9.96.66:44687
I20260806 04:57:32.814101 13698 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.96.66:44687 every 8 connection(s)
I20260806 04:57:32.815327 13500 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785992227050689-9601-0/minicluster-data/ts-1/data/info.pb
I20260806 04:57:32.825363 9601 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu as pid 13500
I20260806 04:57:32.825563 9601 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785992227050689-9601-0/minicluster-data/ts-1/wal/instance
I20260806 04:57:32.826165 9601 external_mini_cluster.cc:1428] Setting key 6675e716cf10f3e7cdbd81e4e7913216
Aug 06 04:57:32 dist-test-slave-x4qh krb5kdc[13138](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992252, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.96.66@KRBTEST.COM for kudu/127.9.96.126@KRBTEST.COM
I20260806 04:57:32.837113 13496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:57:32.817844 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.96.66:52465 (local address 127.9.96.126:35333)
0806 04:57:32.818070 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0806 04:57:32.818077 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 04:57:32.819225 (+ 1148us) server_negotiation.cc:407] Connection header received
0806 04:57:32.821236 (+ 2011us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:57:32.821240 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:57:32.821313 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:57:32.821405 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:57:32.822936 (+ 1531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:57:32.823745 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:57:32.825325 (+ 1580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:57:32.826013 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:57:32.826151 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:57:32.829729 (+ 3578us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:57:32.829738 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:57:32.829746 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:57:32.829790 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:57:32.833472 (+ 3682us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:57:32.834400 (+ 928us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:57:32.834407 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:57:32.834413 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:57:32.834596 (+ 183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:57:32.835087 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:57:32.835095 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:57:32.835101 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:57:32.835515 (+ 414us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:57:32.835668 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0806 04:57:32.836419 (+ 751us) server_negotiation.cc:299] Negotiation successful
0806 04:57:32.836689 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260806 04:57:32.837895 13702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:57:32.818334 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:35333 (local address 127.9.96.66:52465)
0806 04:57:32.819057 (+ 723us) negotiation.cc:107] Waiting for socket to connect
0806 04:57:32.819098 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0806 04:57:32.820988 (+ 1890us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:57:32.821673 (+ 685us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:57:32.821682 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:57:32.822210 (+ 528us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:57:32.822773 (+ 563us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:57:32.822792 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:57:32.823890 (+ 1098us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:57:32.823894 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:57:32.825174 (+ 1280us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:57:32.825187 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:57:32.826256 (+ 1069us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:57:32.826266 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:57:32.826486 (+ 220us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:57:32.827220 (+ 734us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:57:32.827266 (+ 46us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:57:32.829506 (+ 2240us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 04:57:32.833634 (+ 4128us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:57:32.833642 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:57:32.833655 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:57:32.834274 (+ 619us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:57:32.834732 (+ 458us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:57:32.834749 (+ 17us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:57:32.834751 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:57:32.834963 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:57:32.835649 (+ 686us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 04:57:32.835655 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 04:57:32.835995 (+ 340us) client_negotiation.cc:769] Sending connection context
0806 04:57:32.836284 (+ 289us) client_negotiation.cc:240] Negotiation successful
0806 04:57:32.836670 (+ 386us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":497,"thread_start_us":277,"threads_started":1}
I20260806 04:57:32.839506 13699 heartbeater.cc:347] Connected to a master server at 127.9.96.126:35333
I20260806 04:57:32.839869 13699 heartbeater.cc:464] Registering TS with master...
I20260806 04:57:32.840821 13699 heartbeater.cc:511] Master 127.9.96.126:35333 requested a full tablet report, sending...
I20260806 04:57:32.842478 13208 ts_manager.cc:194] Registered new tserver with Master: 7512054f59a34dc19913a5deb9b1b8b3 (127.9.96.66:44687)
I20260806 04:57:32.843820 13208 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.96.66@KRBTEST.COM'} at 127.9.96.66:52465
WARNING: no policy specified for kudu/127.9.96.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.96.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.96.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.67.keytab.
Entry for principal kudu/127.9.96.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.67.keytab.
I20260806 04:57:32.875278 9601 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
/tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785992227050689-9601-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785992227050689-9601-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785992227050689-9601-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785992227050689-9601-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.96.67:0
--local_ip_for_outbound_sockets=127.9.96.67
--webserver_interface=127.9.96.67
--webserver_port=0
--tserver_master_addrs=127.9.96.126:35333
--keytab_file=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.67.keytab
--principal=kudu/127.9.96.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.96.84:37793
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785992227050689-9601-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785992227050689-9601-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785992227050689-9601-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 04:57:33.195878 13705 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 04:57:33.196164 13705 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 04:57:33.196374 13705 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 04:57:33.203887 13705 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 04:57:33.203984 13705 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 04:57:33.204180 13705 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.96.67
I20260806 04:57:33.213534 13705 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.96.84:37793
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785992227050689-9601-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785992227050689-9601-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.67.keytab
--principal=kudu/127.9.96.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.96.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785992227050689-9601-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.96.67
--webserver_port=0
--tserver_master_addrs=127.9.96.126:35333
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.96.67
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785992227050689-9601-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 03:56:12 UTC on ccycloud.quasar-guijbm.root.comops.site
build id 4569618
ASAN enabled
I20260806 04:57:33.215080 13705 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 04:57:33.216427 13705 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 04:57:33.229610 13717 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:57:33.229643 13714 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:57:33.229573 13715 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:57:33.230368 13705 server_base.cc:1034] running on GCE node
I20260806 04:57:33.231014 13705 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 04:57:33.231977 13705 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 04:57:33.233253 13705 hybrid_clock.cc:648] HybridClock initialized: now 1785992253233172 us; error 43 us; skew 500 ppm
I20260806 04:57:33.233577 13705 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 04:57:33 dist-test-slave-x4qh krb5kdc[13138](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992253, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.96.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 04:57:33.236704 13705 init.cc:377] Logged in from keytab as kudu/127.9.96.67@KRBTEST.COM (short username kudu)
I20260806 04:57:33.238021 13705 webserver.cc:460] Webserver started at http://127.9.96.67:35145/ using document root <none> and password file <none>
I20260806 04:57:33.238610 13705 fs_manager.cc:359] Metadata directory not provided
I20260806 04:57:33.238705 13705 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 04:57:33.238952 13705 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 04:57:33.241652 13705 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785992227050689-9601-0/minicluster-data/ts-2/data/instance:
uuid: "010d49b901334790ad345107e631d644"
format_stamp: "Formatted at 2026-08-06 04:57:33 on dist-test-slave-x4qh"
server_key: "1a254e3f40a27907efd557fbbea64a4e"
server_key_iv: "7550444eafbb6e84266c2d43ba461da1"
server_key_version: "encryptionkey@0"
I20260806 04:57:33.242333 13705 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785992227050689-9601-0/minicluster-data/ts-2/wal/instance:
uuid: "010d49b901334790ad345107e631d644"
format_stamp: "Formatted at 2026-08-06 04:57:33 on dist-test-slave-x4qh"
server_key: "1a254e3f40a27907efd557fbbea64a4e"
server_key_iv: "7550444eafbb6e84266c2d43ba461da1"
server_key_version: "encryptionkey@0"
I20260806 04:57:33.247128 13705 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.005s
I20260806 04:57:33.250378 13725 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 04:57:33.251660 13705 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 04:57:33.251796 13705 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785992227050689-9601-0/minicluster-data/ts-2/data,/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785992227050689-9601-0/minicluster-data/ts-2/wal
uuid: "010d49b901334790ad345107e631d644"
format_stamp: "Formatted at 2026-08-06 04:57:33 on dist-test-slave-x4qh"
server_key: "1a254e3f40a27907efd557fbbea64a4e"
server_key_iv: "7550444eafbb6e84266c2d43ba461da1"
server_key_version: "encryptionkey@0"
I20260806 04:57:33.251943 13705 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785992227050689-9601-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785992227050689-9601-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785992227050689-9601-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 04:57:33.265617 13705 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 04:57:33.269066 13705 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 04:57:33.269343 13705 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 04:57:33.270195 13705 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 04:57:33.271477 13705 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 04:57:33.271579 13705 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 04:57:33.271732 13705 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 04:57:33.271821 13705 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 04:57:33.304737 13705 rpc_server.cc:307] RPC server started. Bound to: 127.9.96.67:40593
I20260806 04:57:33.304771 13909 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.96.67:40593 every 8 connection(s)
I20260806 04:57:33.306001 13705 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785992227050689-9601-0/minicluster-data/ts-2/data/info.pb
I20260806 04:57:33.315614 9601 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu as pid 13705
I20260806 04:57:33.315807 9601 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785992227050689-9601-0/minicluster-data/ts-2/wal/instance
I20260806 04:57:33.316367 9601 external_mini_cluster.cc:1428] Setting key 300f64156a88532dc5ff7dd1948c6064
Aug 06 04:57:33 dist-test-slave-x4qh krb5kdc[13138](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992253, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.96.67@KRBTEST.COM for kudu/127.9.96.126@KRBTEST.COM
I20260806 04:57:33.326562 13496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:57:33.308588 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.96.67:59453 (local address 127.9.96.126:35333)
0806 04:57:33.308804 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0806 04:57:33.308810 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 04:57:33.310046 (+ 1236us) server_negotiation.cc:407] Connection header received
0806 04:57:33.311914 (+ 1868us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:57:33.311920 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:57:33.312034 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:57:33.312162 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:57:33.313564 (+ 1402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:57:33.314346 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:57:33.316216 (+ 1870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:57:33.316669 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:57:33.316789 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:57:33.321079 (+ 4290us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:57:33.321084 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:57:33.321087 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:57:33.321120 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:57:33.323899 (+ 2779us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:57:33.324557 (+ 658us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:57:33.324561 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:57:33.324564 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:57:33.324693 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:57:33.325100 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:57:33.325104 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:57:33.325106 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:57:33.325415 (+ 309us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:57:33.325538 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0806 04:57:33.326155 (+ 617us) server_negotiation.cc:299] Negotiation successful
0806 04:57:33.326323 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260806 04:57:33.327478 13912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:57:33.309042 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:35333 (local address 127.9.96.67:59453)
0806 04:57:33.309835 (+ 793us) negotiation.cc:107] Waiting for socket to connect
0806 04:57:33.309900 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0806 04:57:33.311630 (+ 1730us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:57:33.312375 (+ 745us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:57:33.312382 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:57:33.312846 (+ 464us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:57:33.313392 (+ 546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:57:33.313408 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:57:33.314479 (+ 1071us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:57:33.314483 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:57:33.316066 (+ 1583us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:57:33.316082 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:57:33.316930 (+ 848us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:57:33.316939 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:57:33.317236 (+ 297us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:57:33.318280 (+ 1044us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:57:33.318313 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:57:33.320868 (+ 2555us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 04:57:33.324009 (+ 3141us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:57:33.324016 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:57:33.324025 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:57:33.324437 (+ 412us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:57:33.324790 (+ 353us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:57:33.324793 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:57:33.324795 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:57:33.324968 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:57:33.325527 (+ 559us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 04:57:33.325532 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 04:57:33.325839 (+ 307us) client_negotiation.cc:769] Sending connection context
0806 04:57:33.326056 (+ 217us) client_negotiation.cc:240] Negotiation successful
0806 04:57:33.326411 (+ 355us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":528,"thread_start_us":340,"threads_started":1}
I20260806 04:57:33.328755 13910 heartbeater.cc:347] Connected to a master server at 127.9.96.126:35333
I20260806 04:57:33.329001 13910 heartbeater.cc:464] Registering TS with master...
I20260806 04:57:33.329648 13910 heartbeater.cc:511] Master 127.9.96.126:35333 requested a full tablet report, sending...
I20260806 04:57:33.331215 13208 ts_manager.cc:194] Registered new tserver with Master: 010d49b901334790ad345107e631d644 (127.9.96.67:40593)
I20260806 04:57:33.332084 9601 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 04:57:33.332163 13208 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.96.67@KRBTEST.COM'} at 127.9.96.67:59453
I20260806 04:57:33.343086 13494 heartbeater.cc:503] Master 127.9.96.126:35333 was elected leader, sending a full tablet report...
W20260806 04:57:33.366888 13496 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 04:57:33.364508 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43572 (local address 127.9.96.126:35333)
0806 04:57:33.364756 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0806 04:57:33.364765 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 04:57:33.365136 (+ 371us) server_negotiation.cc:407] Connection header received
0806 04:57:33.365269 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:57:33.365273 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:57:33.365355 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:57:33.365441 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:57:33.366605 (+ 1164us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43572: BlockingRecv error: recv got EOF from 127.0.0.1:43572 (error 108)
Metrics: {"server-negotiator.queue_time_us":77}
I20260806 04:57:33.370064 9601 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu pid 13295
W20260806 04:57:33.372871 9601 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu pid 13295: Not authorized: Client connection negotiation failed: client connection to 127.9.96.65:44221: server requires authentication, but client does not have Kerberos credentials available
I20260806 04:57:33.373063 9601 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu with pid 13295
I20260806 04:57:33.398869 9601 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu pid 13500
W20260806 04:57:33.401410 9601 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu pid 13500: Not authorized: Client connection negotiation failed: client connection to 127.9.96.66:44687: server requires authentication, but client does not have Kerberos credentials available
I20260806 04:57:33.401575 9601 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu with pid 13500
I20260806 04:57:33.424160 9601 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu pid 13705
W20260806 04:57:33.427145 9601 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu pid 13705: Not authorized: Client connection negotiation failed: client connection to 127.9.96.67:40593: server requires authentication, but client does not have Kerberos credentials available
I20260806 04:57:33.427321 9601 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu with pid 13705
I20260806 04:57:33.449023 9601 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu pid 13159
I20260806 04:57:33.450173 13249 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 04:57:33.553959 9601 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu with pid 13159
2026-08-06T04:57:33Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (3740 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785992227050689-9601-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 04:57:33 dist-test-slave-x4qh krb5kdc[13937](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 04:57:33 dist-test-slave-x4qh krb5kdc[13937](info): set up 2 sockets
Aug 06 04:57:33 dist-test-slave-x4qh krb5kdc[13937](info): commencing operation
krb5kdc: starting...
W20260806 04:57:34.891776 9601 mini_kdc.cc:121] Time spent starting KDC: real 1.289s user 0.003s sys 0.019s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
W20260806 04:57:35.030439 9601 mini_kdc.cc:253] Time spent creating user principal test-admin: real 0.138s user 0.002s sys 0.006s
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
W20260806 04:57:35.246009 9601 mini_kdc.cc:253] Time spent creating user principal test-user: real 0.215s user 0.000s sys 0.007s
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
W20260806 04:57:35.381732 9601 mini_kdc.cc:253] Time spent creating user principal joe-interloper: real 0.136s user 0.000s sys 0.007s
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785992227050689-9601-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785992227050689-9601-0/krb5kdc/test-user.keytab.
W20260806 04:57:35.816167 9601 mini_kdc.cc:302] Time spent creating keytab for test-user: real 0.434s user 0.000s sys 0.008s
Aug 06 04:57:35 dist-test-slave-x4qh krb5kdc[13937](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992255, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T04:57:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T04:57:35Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.96.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.96.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.96.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.126.keytab.
Entry for principal kudu/127.9.96.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.126.keytab.
W20260806 04:57:36.478504 9601 mini_kdc.cc:262] Time spent creating service keytab for kudu/127.9.96.126: real 0.604s user 0.001s sys 0.010s
I20260806 04:57:36.479691 9601 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
/tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785992227050689-9601-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785992227050689-9601-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785992227050689-9601-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785992227050689-9601-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.96.126:39835
--webserver_interface=127.9.96.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.126.keytab
--principal=kudu/127.9.96.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.96.84:46621
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.96.126:39835
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785992227050689-9601-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785992227050689-9601-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785992227050689-9601-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 04:57:36.802107 13960 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 04:57:36.802439 13960 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 04:57:36.802606 13960 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 04:57:36.809741 13960 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 04:57:36.809883 13960 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 04:57:36.810004 13960 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 04:57:36.810103 13960 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 04:57:36.810197 13960 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 04:57:36.819672 13960 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.96.84:46621
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785992227050689-9601-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785992227050689-9601-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.96.126:39835
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.126.keytab
--principal=kudu/127.9.96.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.96.126:39835
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785992227050689-9601-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.96.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785992227050689-9601-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 03:56:12 UTC on ccycloud.quasar-guijbm.root.comops.site
build id 4569618
ASAN enabled
I20260806 04:57:36.820967 13960 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 04:57:36.822292 13960 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 04:57:36.834127 13965 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:57:36.834172 13971 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:57:36.834209 13973 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:57:36.834981 13960 server_base.cc:1034] running on GCE node
I20260806 04:57:36.835770 13960 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 04:57:36.837024 13960 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 04:57:36.838233 13960 hybrid_clock.cc:648] HybridClock initialized: now 1785992256838188 us; error 33 us; skew 500 ppm
I20260806 04:57:36.838532 13960 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 04:57:36 dist-test-slave-x4qh krb5kdc[13937](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992256, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.96.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 04:57:36.841542 13960 init.cc:377] Logged in from keytab as kudu/127.9.96.126@KRBTEST.COM (short username kudu)
I20260806 04:57:36.842900 13960 webserver.cc:460] Webserver started at http://127.9.96.126:34355/ using document root <none> and password file <none>
I20260806 04:57:36.843576 13960 fs_manager.cc:359] Metadata directory not provided
I20260806 04:57:36.843685 13960 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 04:57:36.843940 13960 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 04:57:36.847258 13960 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785992227050689-9601-0/minicluster-data/master-0/data/instance:
uuid: "50b6a91b330b476585333513ad11295b"
format_stamp: "Formatted at 2026-08-06 04:57:36 on dist-test-slave-x4qh"
server_key: "91107572a879f75cadfce16a19324130"
server_key_iv: "8cd0077164d67d25792115247fec85aa"
server_key_version: "encryptionkey@0"
I20260806 04:57:36.848169 13960 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785992227050689-9601-0/minicluster-data/master-0/wal/instance:
uuid: "50b6a91b330b476585333513ad11295b"
format_stamp: "Formatted at 2026-08-06 04:57:36 on dist-test-slave-x4qh"
server_key: "91107572a879f75cadfce16a19324130"
server_key_iv: "8cd0077164d67d25792115247fec85aa"
server_key_version: "encryptionkey@0"
I20260806 04:57:36.853343 13960 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260806 04:57:36.856904 13986 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 04:57:36.858234 13960 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260806 04:57:36.858376 13960 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785992227050689-9601-0/minicluster-data/master-0/data,/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785992227050689-9601-0/minicluster-data/master-0/wal
uuid: "50b6a91b330b476585333513ad11295b"
format_stamp: "Formatted at 2026-08-06 04:57:36 on dist-test-slave-x4qh"
server_key: "91107572a879f75cadfce16a19324130"
server_key_iv: "8cd0077164d67d25792115247fec85aa"
server_key_version: "encryptionkey@0"
I20260806 04:57:36.858531 13960 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785992227050689-9601-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785992227050689-9601-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785992227050689-9601-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 04:57:36.878190 13960 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 04:57:36.881817 13960 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 04:57:36.882066 13960 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 04:57:36.900617 13960 rpc_server.cc:307] RPC server started. Bound to: 127.9.96.126:39835
I20260806 04:57:36.900660 14061 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.96.126:39835 every 8 connection(s)
I20260806 04:57:36.901959 13960 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785992227050689-9601-0/minicluster-data/master-0/data/info.pb
I20260806 04:57:36.905575 14063 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 04:57:36.910636 9601 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu as pid 13960
I20260806 04:57:36.910878 9601 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785992227050689-9601-0/minicluster-data/master-0/wal/instance
I20260806 04:57:36.911577 9601 external_mini_cluster.cc:1428] Setting key bb3a5f588253dd7687d6cb4033186b1a
I20260806 04:57:36.916692 14063 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 50b6a91b330b476585333513ad11295b: Bootstrap starting.
I20260806 04:57:36.920353 14063 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 50b6a91b330b476585333513ad11295b: Neither blocks nor log segments found. Creating new log.
I20260806 04:57:36.921551 14063 log.cc:824] T 00000000000000000000000000000000 P 50b6a91b330b476585333513ad11295b: Log is configured to *not* fsync() on all Append() calls
I20260806 04:57:36.924499 14063 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 50b6a91b330b476585333513ad11295b: No bootstrap required, opened a new log
Aug 06 04:57:36 dist-test-slave-x4qh krb5kdc[13937](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992255, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.96.126@KRBTEST.COM
I20260806 04:57:36.934866 14063 raft_consensus.cc:348] T 00000000000000000000000000000000 P 50b6a91b330b476585333513ad11295b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "50b6a91b330b476585333513ad11295b" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 39835 } }
I20260806 04:57:36.935202 14069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:57:36.913440 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51902 (local address 127.9.96.126:39835)
0806 04:57:36.914360 (+ 920us) server_negotiation.cc:206] Beginning negotiation
0806 04:57:36.914373 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 04:57:36.914417 (+ 44us) server_negotiation.cc:407] Connection header received
0806 04:57:36.916175 (+ 1758us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:57:36.916197 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:57:36.916547 (+ 350us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:57:36.916846 (+ 299us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:57:36.918811 (+ 1965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:57:36.920173 (+ 1362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:57:36.921928 (+ 1755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:57:36.922603 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:57:36.922749 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:57:36.926732 (+ 3983us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:57:36.926747 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:57:36.926770 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:57:36.926821 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:57:36.930754 (+ 3933us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:57:36.931414 (+ 660us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:57:36.931422 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:57:36.931431 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:57:36.931665 (+ 234us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:57:36.932051 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:57:36.932056 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:57:36.932059 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:57:36.932808 (+ 749us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:57:36.933110 (+ 302us) server_negotiation.cc:1035] Waiting for connection context
0806 04:57:36.933440 (+ 330us) server_negotiation.cc:299] Negotiation successful
0806 04:57:36.933902 (+ 462us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":663,"thread_start_us":408,"threads_started":1}
I20260806 04:57:36.935314 14063 raft_consensus.cc:374] T 00000000000000000000000000000000 P 50b6a91b330b476585333513ad11295b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 04:57:36.935452 14063 raft_consensus.cc:729] T 00000000000000000000000000000000 P 50b6a91b330b476585333513ad11295b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 50b6a91b330b476585333513ad11295b, State: Initialized, Role: FOLLOWER
I20260806 04:57:36.936044 14063 consensus_queue.cc:260] T 00000000000000000000000000000000 P 50b6a91b330b476585333513ad11295b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "50b6a91b330b476585333513ad11295b" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 39835 } }
I20260806 04:57:36.936231 14063 raft_consensus.cc:388] T 00000000000000000000000000000000 P 50b6a91b330b476585333513ad11295b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 04:57:36.936326 14063 raft_consensus.cc:482] T 00000000000000000000000000000000 P 50b6a91b330b476585333513ad11295b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 04:57:36.936533 14063 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 50b6a91b330b476585333513ad11295b [term 0 FOLLOWER]: Advancing to term 1
I20260806 04:57:36.938867 9601 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 04:57:36.939095 14063 raft_consensus.cc:504] T 00000000000000000000000000000000 P 50b6a91b330b476585333513ad11295b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "50b6a91b330b476585333513ad11295b" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 39835 } }
I20260806 04:57:36.939639 14063 leader_election.cc:302] T 00000000000000000000000000000000 P 50b6a91b330b476585333513ad11295b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 50b6a91b330b476585333513ad11295b; no voters:
I20260806 04:57:36.940214 14063 leader_election.cc:288] T 00000000000000000000000000000000 P 50b6a91b330b476585333513ad11295b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 04:57:36.940394 14071 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 50b6a91b330b476585333513ad11295b [term 1 FOLLOWER]: Leader election won for term 1
I20260806 04:57:36.941059 14071 raft_consensus.cc:686] T 00000000000000000000000000000000 P 50b6a91b330b476585333513ad11295b [term 1 LEADER]: Becoming Leader. State: Replica: 50b6a91b330b476585333513ad11295b, State: Running, Role: LEADER
I20260806 04:57:36.941677 14071 consensus_queue.cc:237] T 00000000000000000000000000000000 P 50b6a91b330b476585333513ad11295b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "50b6a91b330b476585333513ad11295b" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 39835 } }
I20260806 04:57:36.943609 14071 sys_catalog.cc:455] T 00000000000000000000000000000000 P 50b6a91b330b476585333513ad11295b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 50b6a91b330b476585333513ad11295b. Latest consensus state: current_term: 1 leader_uuid: "50b6a91b330b476585333513ad11295b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "50b6a91b330b476585333513ad11295b" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 39835 } } }
I20260806 04:57:36.943962 14071 sys_catalog.cc:458] T 00000000000000000000000000000000 P 50b6a91b330b476585333513ad11295b [sys.catalog]: This master's current role is: LEADER
I20260806 04:57:36.944586 14074 sys_catalog.cc:455] T 00000000000000000000000000000000 P 50b6a91b330b476585333513ad11295b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "50b6a91b330b476585333513ad11295b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "50b6a91b330b476585333513ad11295b" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 39835 } } }
I20260806 04:57:36.944974 14074 sys_catalog.cc:458] T 00000000000000000000000000000000 P 50b6a91b330b476585333513ad11295b [sys.catalog]: This master's current role is: LEADER
I20260806 04:57:36.951314 14063 sys_catalog.cc:564] T 00000000000000000000000000000000 P 50b6a91b330b476585333513ad11295b [sys.catalog]: configured and running, proceeding with master startup.
I20260806 04:57:36.951572 14078 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 04:57:36.956940 14078 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 04:57:36.966811 14078 catalog_manager.cc:1347] Generated new cluster ID: b0670fa7acfc4ca59cd391c9bf542bd9
I20260806 04:57:36.966915 14078 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 04:57:36.982831 14078 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 04:57:36.985267 14078 catalog_manager.cc:1504] Loading token signing keys...
I20260806 04:57:36.988472 14102 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260806 04:57:36.992588 14078 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 50b6a91b330b476585333513ad11295b: Generated new TSK 0
I20260806 04:57:36.993582 14078 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 04:57:37.001540 14069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:57:36.992559 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51910 (local address 127.9.96.126:39835)
0806 04:57:36.992754 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0806 04:57:36.992760 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 04:57:36.993695 (+ 935us) server_negotiation.cc:407] Connection header received
0806 04:57:36.993973 (+ 278us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:57:36.993984 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:57:36.994081 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:57:36.994217 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:57:36.995552 (+ 1335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:57:36.996362 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:57:36.998257 (+ 1895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:57:36.998672 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:57:36.998767 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:57:36.999991 (+ 1224us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:57:36.999996 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:57:36.999999 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 04:57:37.000725 (+ 726us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:57:37.000787 (+ 62us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 04:57:37.000871 (+ 84us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:57:37.000979 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0806 04:57:37.001151 (+ 172us) server_negotiation.cc:299] Negotiation successful
0806 04:57:37.001226 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260806 04:57:37.010021 9601 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu pid 13960
I20260806 04:57:37.011222 14056 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 04:57:37.097338 9601 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu with pid 13960
2026-08-06T04:57:37Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (3540 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785992227050689-9601-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 04:57:37 dist-test-slave-x4qh krb5kdc[14122](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 04:57:37 dist-test-slave-x4qh krb5kdc[14122](info): set up 2 sockets
Aug 06 04:57:37 dist-test-slave-x4qh krb5kdc[14122](info): commencing operation
krb5kdc: starting...
W20260806 04:57:38.367537 9601 mini_kdc.cc:121] Time spent starting KDC: real 1.224s user 0.003s sys 0.019s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785992227050689-9601-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785992227050689-9601-0/krb5kdc/test-user.keytab.
Aug 06 04:57:38 dist-test-slave-x4qh krb5kdc[14122](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992258, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T04:57:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T04:57:38Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.96.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.96.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.96.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.126.keytab.
Entry for principal kudu/127.9.96.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.126.keytab.
I20260806 04:57:38.567080 9601 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
/tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785992227050689-9601-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785992227050689-9601-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785992227050689-9601-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785992227050689-9601-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.96.126:43727
--webserver_interface=127.9.96.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.126.keytab
--principal=kudu/127.9.96.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.96.84:37395
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.96.126:43727
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785992227050689-9601-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785992227050689-9601-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785992227050689-9601-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 04:57:38.889768 14142 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 04:57:38.890100 14142 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 04:57:38.890273 14142 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 04:57:38.897589 14142 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 04:57:38.897723 14142 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 04:57:38.897840 14142 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 04:57:38.897940 14142 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 04:57:38.898030 14142 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 04:57:38.907744 14142 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.96.84:37395
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785992227050689-9601-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785992227050689-9601-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.96.126:43727
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.126.keytab
--principal=kudu/127.9.96.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.96.126:43727
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785992227050689-9601-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.96.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785992227050689-9601-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 03:56:12 UTC on ccycloud.quasar-guijbm.root.comops.site
build id 4569618
ASAN enabled
I20260806 04:57:38.909055 14142 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 04:57:38.910390 14142 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 04:57:38.922506 14151 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:57:38.922518 14154 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:57:38.922746 14157 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:57:38.923534 14142 server_base.cc:1034] running on GCE node
I20260806 04:57:38.924376 14142 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 04:57:38.925757 14142 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 04:57:38.926990 14142 hybrid_clock.cc:648] HybridClock initialized: now 1785992258926962 us; error 47 us; skew 500 ppm
I20260806 04:57:38.927322 14142 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 04:57:38 dist-test-slave-x4qh krb5kdc[14122](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992258, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.96.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 04:57:38.934758 14142 init.cc:377] Logged in from keytab as kudu/127.9.96.126@KRBTEST.COM (short username kudu)
I20260806 04:57:38.936349 14142 webserver.cc:460] Webserver started at http://127.9.96.126:37687/ using document root <none> and password file <none>
I20260806 04:57:38.937036 14142 fs_manager.cc:359] Metadata directory not provided
I20260806 04:57:38.937193 14142 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 04:57:38.937512 14142 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 04:57:38.940502 14142 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785992227050689-9601-0/minicluster-data/master-0/data/instance:
uuid: "5c972b0d7a1c4f5197efb4ed8930a6ec"
format_stamp: "Formatted at 2026-08-06 04:57:38 on dist-test-slave-x4qh"
server_key: "ad961c9a606c8041b87799e06030f37a"
server_key_iv: "55cc391aa90438722d772179a6eda963"
server_key_version: "encryptionkey@0"
I20260806 04:57:38.941247 14142 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785992227050689-9601-0/minicluster-data/master-0/wal/instance:
uuid: "5c972b0d7a1c4f5197efb4ed8930a6ec"
format_stamp: "Formatted at 2026-08-06 04:57:38 on dist-test-slave-x4qh"
server_key: "ad961c9a606c8041b87799e06030f37a"
server_key_iv: "55cc391aa90438722d772179a6eda963"
server_key_version: "encryptionkey@0"
I20260806 04:57:38.946422 14142 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260806 04:57:38.949713 14165 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 04:57:38.950917 14142 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 04:57:38.951104 14142 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785992227050689-9601-0/minicluster-data/master-0/data,/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785992227050689-9601-0/minicluster-data/master-0/wal
uuid: "5c972b0d7a1c4f5197efb4ed8930a6ec"
format_stamp: "Formatted at 2026-08-06 04:57:38 on dist-test-slave-x4qh"
server_key: "ad961c9a606c8041b87799e06030f37a"
server_key_iv: "55cc391aa90438722d772179a6eda963"
server_key_version: "encryptionkey@0"
I20260806 04:57:38.951323 14142 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785992227050689-9601-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785992227050689-9601-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785992227050689-9601-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 04:57:38.972993 14142 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 04:57:38.976744 14142 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 04:57:38.977036 14142 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 04:57:38.995565 14142 rpc_server.cc:307] RPC server started. Bound to: 127.9.96.126:43727
I20260806 04:57:38.995584 14252 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.96.126:43727 every 8 connection(s)
I20260806 04:57:38.996888 14142 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785992227050689-9601-0/minicluster-data/master-0/data/info.pb
I20260806 04:57:39.000470 14253 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 04:57:39.005643 9601 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu as pid 14142
I20260806 04:57:39.005923 9601 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785992227050689-9601-0/minicluster-data/master-0/wal/instance
I20260806 04:57:39.006585 9601 external_mini_cluster.cc:1428] Setting key 87bc36b04a46aa6b925db3ca4a1ad950
I20260806 04:57:39.012568 14253 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5c972b0d7a1c4f5197efb4ed8930a6ec: Bootstrap starting.
I20260806 04:57:39.016317 14253 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5c972b0d7a1c4f5197efb4ed8930a6ec: Neither blocks nor log segments found. Creating new log.
I20260806 04:57:39.017405 14253 log.cc:824] T 00000000000000000000000000000000 P 5c972b0d7a1c4f5197efb4ed8930a6ec: Log is configured to *not* fsync() on all Append() calls
I20260806 04:57:39.020421 14253 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5c972b0d7a1c4f5197efb4ed8930a6ec: No bootstrap required, opened a new log
Aug 06 04:57:39 dist-test-slave-x4qh krb5kdc[14122](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992258, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.96.126@KRBTEST.COM
I20260806 04:57:39.030355 14253 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5c972b0d7a1c4f5197efb4ed8930a6ec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c972b0d7a1c4f5197efb4ed8930a6ec" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 43727 } }
I20260806 04:57:39.030611 14256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:57:39.008743 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60310 (local address 127.9.96.126:43727)
0806 04:57:39.009672 (+ 929us) server_negotiation.cc:206] Beginning negotiation
0806 04:57:39.009687 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 04:57:39.009748 (+ 61us) server_negotiation.cc:407] Connection header received
0806 04:57:39.012164 (+ 2416us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:57:39.012194 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:57:39.012689 (+ 495us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:57:39.013139 (+ 450us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:57:39.014486 (+ 1347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:57:39.016526 (+ 2040us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:57:39.017850 (+ 1324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:57:39.018718 (+ 868us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:57:39.018904 (+ 186us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:57:39.023299 (+ 4395us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:57:39.023307 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:57:39.023320 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:57:39.023362 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:57:39.026629 (+ 3267us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:57:39.027473 (+ 844us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:57:39.027479 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:57:39.027484 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:57:39.027651 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:57:39.028057 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:57:39.028060 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:57:39.028062 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:57:39.028577 (+ 515us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:57:39.028811 (+ 234us) server_negotiation.cc:1035] Waiting for connection context
0806 04:57:39.029289 (+ 478us) server_negotiation.cc:299] Negotiation successful
0806 04:57:39.029643 (+ 354us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":664,"thread_start_us":415,"threads_started":1}
I20260806 04:57:39.030723 14253 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5c972b0d7a1c4f5197efb4ed8930a6ec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 04:57:39.030826 14253 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5c972b0d7a1c4f5197efb4ed8930a6ec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c972b0d7a1c4f5197efb4ed8930a6ec, State: Initialized, Role: FOLLOWER
I20260806 04:57:39.031492 14253 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5c972b0d7a1c4f5197efb4ed8930a6ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c972b0d7a1c4f5197efb4ed8930a6ec" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 43727 } }
I20260806 04:57:39.031749 14253 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5c972b0d7a1c4f5197efb4ed8930a6ec [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 04:57:39.031898 14253 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5c972b0d7a1c4f5197efb4ed8930a6ec [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 04:57:39.032087 14253 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5c972b0d7a1c4f5197efb4ed8930a6ec [term 0 FOLLOWER]: Advancing to term 1
I20260806 04:57:39.034557 14253 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5c972b0d7a1c4f5197efb4ed8930a6ec [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c972b0d7a1c4f5197efb4ed8930a6ec" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 43727 } }
I20260806 04:57:39.035027 14253 leader_election.cc:302] T 00000000000000000000000000000000 P 5c972b0d7a1c4f5197efb4ed8930a6ec [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5c972b0d7a1c4f5197efb4ed8930a6ec; no voters:
I20260806 04:57:39.035631 14253 leader_election.cc:288] T 00000000000000000000000000000000 P 5c972b0d7a1c4f5197efb4ed8930a6ec [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 04:57:39.035776 14259 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5c972b0d7a1c4f5197efb4ed8930a6ec [term 1 FOLLOWER]: Leader election won for term 1
I20260806 04:57:39.036463 14259 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5c972b0d7a1c4f5197efb4ed8930a6ec [term 1 LEADER]: Becoming Leader. State: Replica: 5c972b0d7a1c4f5197efb4ed8930a6ec, State: Running, Role: LEADER
I20260806 04:57:39.037038 14259 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5c972b0d7a1c4f5197efb4ed8930a6ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c972b0d7a1c4f5197efb4ed8930a6ec" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 43727 } }
I20260806 04:57:39.037699 14253 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5c972b0d7a1c4f5197efb4ed8930a6ec [sys.catalog]: configured and running, proceeding with master startup.
I20260806 04:57:39.039805 14263 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5c972b0d7a1c4f5197efb4ed8930a6ec [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5c972b0d7a1c4f5197efb4ed8930a6ec. Latest consensus state: current_term: 1 leader_uuid: "5c972b0d7a1c4f5197efb4ed8930a6ec" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c972b0d7a1c4f5197efb4ed8930a6ec" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 43727 } } }
I20260806 04:57:39.040134 14263 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5c972b0d7a1c4f5197efb4ed8930a6ec [sys.catalog]: This master's current role is: LEADER
I20260806 04:57:39.040652 14262 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5c972b0d7a1c4f5197efb4ed8930a6ec [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5c972b0d7a1c4f5197efb4ed8930a6ec" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c972b0d7a1c4f5197efb4ed8930a6ec" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 43727 } } }
I20260806 04:57:39.040980 14262 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5c972b0d7a1c4f5197efb4ed8930a6ec [sys.catalog]: This master's current role is: LEADER
I20260806 04:57:39.041148 14268 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 04:57:39.046528 14268 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 04:57:39.056360 14268 catalog_manager.cc:1347] Generated new cluster ID: 82fb8c7ca97343bfa8fdf6253523af0e
I20260806 04:57:39.056465 14268 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 04:57:39.065439 14268 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 04:57:39.067114 14268 catalog_manager.cc:1504] Loading token signing keys...
I20260806 04:57:39.075067 14268 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5c972b0d7a1c4f5197efb4ed8930a6ec: Generated new TSK 0
I20260806 04:57:39.075942 14268 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.9.96.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.96.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.96.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.65.keytab.
Entry for principal kudu/127.9.96.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.65.keytab.
I20260806 04:57:39.126470 9601 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
/tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785992227050689-9601-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785992227050689-9601-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785992227050689-9601-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785992227050689-9601-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.96.65:0
--local_ip_for_outbound_sockets=127.9.96.65
--webserver_interface=127.9.96.65
--webserver_port=0
--tserver_master_addrs=127.9.96.126:43727
--keytab_file=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.65.keytab
--principal=kudu/127.9.96.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.96.84:37395
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785992227050689-9601-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785992227050689-9601-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785992227050689-9601-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 04:57:39.461426 14294 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 04:57:39.461685 14294 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 04:57:39.461807 14294 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 04:57:39.468911 14294 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 04:57:39.469017 14294 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 04:57:39.469136 14294 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.96.65
I20260806 04:57:39.478623 14294 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.96.84:37395
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785992227050689-9601-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785992227050689-9601-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.65.keytab
--principal=kudu/127.9.96.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.96.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785992227050689-9601-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.96.65
--webserver_port=0
--tserver_master_addrs=127.9.96.126:43727
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.96.65
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785992227050689-9601-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 03:56:12 UTC on ccycloud.quasar-guijbm.root.comops.site
build id 4569618
ASAN enabled
I20260806 04:57:39.479848 14294 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 04:57:39.481168 14294 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 04:57:39.494737 14305 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:57:39.494859 14301 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:57:39.494912 14303 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:57:39.495908 14294 server_base.cc:1034] running on GCE node
I20260806 04:57:39.496554 14294 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 04:57:39.497432 14294 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 04:57:39.498660 14294 hybrid_clock.cc:648] HybridClock initialized: now 1785992259498609 us; error 43 us; skew 500 ppm
I20260806 04:57:39.498980 14294 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 04:57:39 dist-test-slave-x4qh krb5kdc[14122](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992259, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.96.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 04:57:39.502046 14294 init.cc:377] Logged in from keytab as kudu/127.9.96.65@KRBTEST.COM (short username kudu)
I20260806 04:57:39.503484 14294 webserver.cc:460] Webserver started at http://127.9.96.65:36539/ using document root <none> and password file <none>
I20260806 04:57:39.504134 14294 fs_manager.cc:359] Metadata directory not provided
I20260806 04:57:39.504282 14294 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 04:57:39.504601 14294 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 04:57:39.508554 14294 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785992227050689-9601-0/minicluster-data/ts-0/data/instance:
uuid: "4be6ea757b854935a4c4bf48979c1aa6"
format_stamp: "Formatted at 2026-08-06 04:57:39 on dist-test-slave-x4qh"
server_key: "91c6ad714f3d60152190072010d65d7a"
server_key_iv: "7ffafae8160f444c1b3729dbabbf5ec4"
server_key_version: "encryptionkey@0"
I20260806 04:57:39.509680 14294 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785992227050689-9601-0/minicluster-data/ts-0/wal/instance:
uuid: "4be6ea757b854935a4c4bf48979c1aa6"
format_stamp: "Formatted at 2026-08-06 04:57:39 on dist-test-slave-x4qh"
server_key: "91c6ad714f3d60152190072010d65d7a"
server_key_iv: "7ffafae8160f444c1b3729dbabbf5ec4"
server_key_version: "encryptionkey@0"
I20260806 04:57:39.519666 14294 fs_manager.cc:691] Time spent creating directory manager: real 0.009s user 0.006s sys 0.005s
I20260806 04:57:39.523543 14315 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 04:57:39.524823 14294 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 04:57:39.524991 14294 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785992227050689-9601-0/minicluster-data/ts-0/data,/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785992227050689-9601-0/minicluster-data/ts-0/wal
uuid: "4be6ea757b854935a4c4bf48979c1aa6"
format_stamp: "Formatted at 2026-08-06 04:57:39 on dist-test-slave-x4qh"
server_key: "91c6ad714f3d60152190072010d65d7a"
server_key_iv: "7ffafae8160f444c1b3729dbabbf5ec4"
server_key_version: "encryptionkey@0"
I20260806 04:57:39.525194 14294 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785992227050689-9601-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785992227050689-9601-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785992227050689-9601-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 04:57:39.536164 14294 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 04:57:39.539769 14294 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 04:57:39.540081 14294 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 04:57:39.540938 14294 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 04:57:39.542191 14294 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 04:57:39.542269 14294 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 04:57:39.542429 14294 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 04:57:39.542539 14294 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 04:57:39.576455 14294 rpc_server.cc:307] RPC server started. Bound to: 127.9.96.65:34695
I20260806 04:57:39.576506 14479 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.96.65:34695 every 8 connection(s)
I20260806 04:57:39.577719 14294 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785992227050689-9601-0/minicluster-data/ts-0/data/info.pb
I20260806 04:57:39.584975 9601 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu as pid 14294
I20260806 04:57:39.585214 9601 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785992227050689-9601-0/minicluster-data/ts-0/wal/instance
I20260806 04:57:39.585929 9601 external_mini_cluster.cc:1428] Setting key bbec875b65174a3f0bba2d0a3afc7750
Aug 06 04:57:39 dist-test-slave-x4qh krb5kdc[14122](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992259, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.96.65@KRBTEST.COM for kudu/127.9.96.126@KRBTEST.COM
I20260806 04:57:39.599325 14483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:57:39.580138 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.96.65:49745 (local address 127.9.96.126:43727)
0806 04:57:39.580600 (+ 462us) server_negotiation.cc:206] Beginning negotiation
0806 04:57:39.580607 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 04:57:39.581541 (+ 934us) server_negotiation.cc:407] Connection header received
0806 04:57:39.583164 (+ 1623us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:57:39.583168 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:57:39.583234 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:57:39.583329 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:57:39.584907 (+ 1578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:57:39.585759 (+ 852us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:57:39.587809 (+ 2050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:57:39.588264 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:57:39.588363 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:57:39.592568 (+ 4205us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:57:39.592577 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:57:39.592585 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:57:39.592630 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:57:39.595848 (+ 3218us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:57:39.596804 (+ 956us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:57:39.596808 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:57:39.596810 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:57:39.596929 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:57:39.597410 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:57:39.597418 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:57:39.597425 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:57:39.597806 (+ 381us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:57:39.597965 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0806 04:57:39.598595 (+ 630us) server_negotiation.cc:299] Negotiation successful
0806 04:57:39.598856 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":330,"thread_start_us":225,"threads_started":1}
I20260806 04:57:39.599877 14484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:57:39.580655 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:43727 (local address 127.9.96.65:49745)
0806 04:57:39.581350 (+ 695us) negotiation.cc:107] Waiting for socket to connect
0806 04:57:39.581395 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0806 04:57:39.582920 (+ 1525us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:57:39.583569 (+ 649us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:57:39.583577 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:57:39.584054 (+ 477us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:57:39.584717 (+ 663us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:57:39.584746 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:57:39.585936 (+ 1190us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:57:39.585943 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:57:39.587593 (+ 1650us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:57:39.587626 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:57:39.588495 (+ 869us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:57:39.588505 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:57:39.588804 (+ 299us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:57:39.589600 (+ 796us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:57:39.589634 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:57:39.592354 (+ 2720us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 04:57:39.595993 (+ 3639us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:57:39.596002 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:57:39.596018 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:57:39.596681 (+ 663us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:57:39.597028 (+ 347us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:57:39.597032 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:57:39.597034 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:57:39.597262 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:57:39.597934 (+ 672us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 04:57:39.597938 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 04:57:39.598234 (+ 296us) client_negotiation.cc:769] Sending connection context
0806 04:57:39.598433 (+ 199us) client_negotiation.cc:240] Negotiation successful
0806 04:57:39.598788 (+ 355us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":476,"thread_start_us":270,"threads_started":1}
I20260806 04:57:39.601403 14480 heartbeater.cc:347] Connected to a master server at 127.9.96.126:43727
I20260806 04:57:39.601748 14480 heartbeater.cc:464] Registering TS with master...
I20260806 04:57:39.602588 14480 heartbeater.cc:511] Master 127.9.96.126:43727 requested a full tablet report, sending...
I20260806 04:57:39.605132 14190 ts_manager.cc:194] Registered new tserver with Master: 4be6ea757b854935a4c4bf48979c1aa6 (127.9.96.65:34695)
I20260806 04:57:39.607448 14190 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.96.65@KRBTEST.COM'} at 127.9.96.65:49745
WARNING: no policy specified for kudu/127.9.96.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.96.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.96.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.66.keytab.
Entry for principal kudu/127.9.96.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.66.keytab.
I20260806 04:57:39.635772 9601 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
/tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785992227050689-9601-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785992227050689-9601-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785992227050689-9601-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785992227050689-9601-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.96.66:0
--local_ip_for_outbound_sockets=127.9.96.66
--webserver_interface=127.9.96.66
--webserver_port=0
--tserver_master_addrs=127.9.96.126:43727
--keytab_file=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.66.keytab
--principal=kudu/127.9.96.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.96.84:37395
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785992227050689-9601-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785992227050689-9601-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785992227050689-9601-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 04:57:39.954768 14490 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 04:57:39.955020 14490 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 04:57:39.955168 14490 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 04:57:39.962291 14490 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 04:57:39.962396 14490 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 04:57:39.962576 14490 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.96.66
I20260806 04:57:39.971926 14490 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.96.84:37395
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785992227050689-9601-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785992227050689-9601-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.66.keytab
--principal=kudu/127.9.96.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.96.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785992227050689-9601-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.96.66
--webserver_port=0
--tserver_master_addrs=127.9.96.126:43727
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.96.66
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785992227050689-9601-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 03:56:12 UTC on ccycloud.quasar-guijbm.root.comops.site
build id 4569618
ASAN enabled
I20260806 04:57:39.973095 14490 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 04:57:39.974346 14490 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 04:57:39.987680 14496 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:57:39.987763 14497 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:57:39.987749 14501 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:57:39.988673 14490 server_base.cc:1034] running on GCE node
I20260806 04:57:39.989267 14490 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 04:57:39.990159 14490 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 04:57:39.991433 14490 hybrid_clock.cc:648] HybridClock initialized: now 1785992259991353 us; error 50 us; skew 500 ppm
I20260806 04:57:39.991746 14490 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 04:57:39 dist-test-slave-x4qh krb5kdc[14122](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992259, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.96.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 04:57:39.995304 14490 init.cc:377] Logged in from keytab as kudu/127.9.96.66@KRBTEST.COM (short username kudu)
I20260806 04:57:40.005544 14490 webserver.cc:460] Webserver started at http://127.9.96.66:41455/ using document root <none> and password file <none>
I20260806 04:57:40.006345 14490 fs_manager.cc:359] Metadata directory not provided
I20260806 04:57:40.006485 14490 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 04:57:40.006826 14490 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 04:57:40.009935 14490 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785992227050689-9601-0/minicluster-data/ts-1/data/instance:
uuid: "d89aba434a5a41e8ae3cc73c2f51ccef"
format_stamp: "Formatted at 2026-08-06 04:57:40 on dist-test-slave-x4qh"
server_key: "be36c1bdf4207463e8b0cf7ecfb7d25d"
server_key_iv: "16482e693bdea8f82646936ff2c14068"
server_key_version: "encryptionkey@0"
I20260806 04:57:40.010782 14490 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785992227050689-9601-0/minicluster-data/ts-1/wal/instance:
uuid: "d89aba434a5a41e8ae3cc73c2f51ccef"
format_stamp: "Formatted at 2026-08-06 04:57:40 on dist-test-slave-x4qh"
server_key: "be36c1bdf4207463e8b0cf7ecfb7d25d"
server_key_iv: "16482e693bdea8f82646936ff2c14068"
server_key_version: "encryptionkey@0"
I20260806 04:57:40.016420 14490 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.002s
I20260806 04:57:40.020113 14510 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 04:57:40.021693 14490 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 04:57:40.021844 14490 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785992227050689-9601-0/minicluster-data/ts-1/data,/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785992227050689-9601-0/minicluster-data/ts-1/wal
uuid: "d89aba434a5a41e8ae3cc73c2f51ccef"
format_stamp: "Formatted at 2026-08-06 04:57:40 on dist-test-slave-x4qh"
server_key: "be36c1bdf4207463e8b0cf7ecfb7d25d"
server_key_iv: "16482e693bdea8f82646936ff2c14068"
server_key_version: "encryptionkey@0"
I20260806 04:57:40.022027 14490 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785992227050689-9601-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785992227050689-9601-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785992227050689-9601-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 04:57:40.062287 14490 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 04:57:40.066479 14490 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 04:57:40.066780 14490 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 04:57:40.067747 14490 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 04:57:40.069015 14490 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 04:57:40.069095 14490 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 04:57:40.069275 14490 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 04:57:40.069326 14490 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 04:57:40.101648 14490 rpc_server.cc:307] RPC server started. Bound to: 127.9.96.66:35995
I20260806 04:57:40.101734 14674 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.96.66:35995 every 8 connection(s)
I20260806 04:57:40.102895 14490 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785992227050689-9601-0/minicluster-data/ts-1/data/info.pb
I20260806 04:57:40.113246 9601 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu as pid 14490
I20260806 04:57:40.113461 9601 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785992227050689-9601-0/minicluster-data/ts-1/wal/instance
I20260806 04:57:40.114055 9601 external_mini_cluster.cc:1428] Setting key 941ceb97de0a5e49c29ae554e59df877
Aug 06 04:57:40 dist-test-slave-x4qh krb5kdc[14122](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992259, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.96.66@KRBTEST.COM for kudu/127.9.96.126@KRBTEST.COM
I20260806 04:57:40.122185 14677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:57:40.105543 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.96.66:53871 (local address 127.9.96.126:43727)
0806 04:57:40.106039 (+ 496us) server_negotiation.cc:206] Beginning negotiation
0806 04:57:40.106046 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 04:57:40.106970 (+ 924us) server_negotiation.cc:407] Connection header received
0806 04:57:40.108938 (+ 1968us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:57:40.108942 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:57:40.109006 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:57:40.109100 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:57:40.110451 (+ 1351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:57:40.111109 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:57:40.112526 (+ 1417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:57:40.112948 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:57:40.113086 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:57:40.116595 (+ 3509us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:57:40.116599 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:57:40.116603 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:57:40.116648 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:57:40.119152 (+ 2504us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:57:40.119944 (+ 792us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:57:40.119953 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:57:40.119957 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:57:40.120077 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:57:40.120567 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:57:40.120571 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:57:40.120578 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:57:40.120830 (+ 252us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:57:40.120975 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0806 04:57:40.121607 (+ 632us) server_negotiation.cc:299] Negotiation successful
0806 04:57:40.121814 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":330,"thread_start_us":216,"threads_started":1}
I20260806 04:57:40.123119 14678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:57:40.106099 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:43727 (local address 127.9.96.66:53871)
0806 04:57:40.106810 (+ 711us) negotiation.cc:107] Waiting for socket to connect
0806 04:57:40.106854 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0806 04:57:40.108589 (+ 1735us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:57:40.109317 (+ 728us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:57:40.109324 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:57:40.109806 (+ 482us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:57:40.110308 (+ 502us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:57:40.110323 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:57:40.111228 (+ 905us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:57:40.111231 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:57:40.112409 (+ 1178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:57:40.112420 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:57:40.113134 (+ 714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:57:40.113138 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:57:40.113310 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:57:40.114035 (+ 725us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:57:40.114054 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:57:40.116393 (+ 2339us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 04:57:40.119281 (+ 2888us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:57:40.119290 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:57:40.119303 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:57:40.119809 (+ 506us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:57:40.120187 (+ 378us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:57:40.120192 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:57:40.120195 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:57:40.120436 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:57:40.120951 (+ 515us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 04:57:40.120958 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 04:57:40.121270 (+ 312us) client_negotiation.cc:769] Sending connection context
0806 04:57:40.121511 (+ 241us) client_negotiation.cc:240] Negotiation successful
0806 04:57:40.122008 (+ 497us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":480,"thread_start_us":289,"threads_started":1}
I20260806 04:57:40.124569 14675 heartbeater.cc:347] Connected to a master server at 127.9.96.126:43727
I20260806 04:57:40.124859 14675 heartbeater.cc:464] Registering TS with master...
I20260806 04:57:40.125568 14675 heartbeater.cc:511] Master 127.9.96.126:43727 requested a full tablet report, sending...
I20260806 04:57:40.127147 14190 ts_manager.cc:194] Registered new tserver with Master: d89aba434a5a41e8ae3cc73c2f51ccef (127.9.96.66:35995)
I20260806 04:57:40.128428 14190 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.96.66@KRBTEST.COM'} at 127.9.96.66:53871
WARNING: no policy specified for kudu/127.9.96.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.96.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.96.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.67.keytab.
Entry for principal kudu/127.9.96.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.67.keytab.
I20260806 04:57:40.160403 9601 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
/tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785992227050689-9601-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785992227050689-9601-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785992227050689-9601-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785992227050689-9601-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.96.67:0
--local_ip_for_outbound_sockets=127.9.96.67
--webserver_interface=127.9.96.67
--webserver_port=0
--tserver_master_addrs=127.9.96.126:43727
--keytab_file=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.67.keytab
--principal=kudu/127.9.96.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.96.84:37395
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785992227050689-9601-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785992227050689-9601-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785992227050689-9601-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 04:57:40.481803 14683 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 04:57:40.482045 14683 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 04:57:40.482194 14683 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 04:57:40.489140 14683 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 04:57:40.489243 14683 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 04:57:40.489403 14683 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.96.67
I20260806 04:57:40.498669 14683 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.96.84:37395
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785992227050689-9601-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785992227050689-9601-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.67.keytab
--principal=kudu/127.9.96.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.96.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785992227050689-9601-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.96.67
--webserver_port=0
--tserver_master_addrs=127.9.96.126:43727
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.96.67
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785992227050689-9601-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 03:56:12 UTC on ccycloud.quasar-guijbm.root.comops.site
build id 4569618
ASAN enabled
I20260806 04:57:40.499874 14683 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 04:57:40.501061 14683 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 04:57:40.517040 14690 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:57:40.517139 14693 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:57:40.517946 14683 server_base.cc:1034] running on GCE node
W20260806 04:57:40.518045 14689 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:57:40.518767 14683 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 04:57:40.519877 14683 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 04:57:40.521104 14683 hybrid_clock.cc:648] HybridClock initialized: now 1785992260521049 us; error 45 us; skew 500 ppm
I20260806 04:57:40.521572 14683 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 04:57:40 dist-test-slave-x4qh krb5kdc[14122](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992260, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.96.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 04:57:40.525034 14683 init.cc:377] Logged in from keytab as kudu/127.9.96.67@KRBTEST.COM (short username kudu)
I20260806 04:57:40.527076 14683 webserver.cc:460] Webserver started at http://127.9.96.67:44753/ using document root <none> and password file <none>
I20260806 04:57:40.527962 14683 fs_manager.cc:359] Metadata directory not provided
I20260806 04:57:40.528126 14683 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 04:57:40.528537 14683 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 04:57:40.532900 14683 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785992227050689-9601-0/minicluster-data/ts-2/data/instance:
uuid: "7a707dc40ea7417da7b5888478870cb9"
format_stamp: "Formatted at 2026-08-06 04:57:40 on dist-test-slave-x4qh"
server_key: "695f6da957f2fbf88740364d70c30890"
server_key_iv: "90f8924c64c589cf883974b0885662a9"
server_key_version: "encryptionkey@0"
I20260806 04:57:40.533953 14683 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785992227050689-9601-0/minicluster-data/ts-2/wal/instance:
uuid: "7a707dc40ea7417da7b5888478870cb9"
format_stamp: "Formatted at 2026-08-06 04:57:40 on dist-test-slave-x4qh"
server_key: "695f6da957f2fbf88740364d70c30890"
server_key_iv: "90f8924c64c589cf883974b0885662a9"
server_key_version: "encryptionkey@0"
I20260806 04:57:40.540426 14683 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.004s
I20260806 04:57:40.543807 14705 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 04:57:40.545086 14683 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260806 04:57:40.545269 14683 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785992227050689-9601-0/minicluster-data/ts-2/data,/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785992227050689-9601-0/minicluster-data/ts-2/wal
uuid: "7a707dc40ea7417da7b5888478870cb9"
format_stamp: "Formatted at 2026-08-06 04:57:40 on dist-test-slave-x4qh"
server_key: "695f6da957f2fbf88740364d70c30890"
server_key_iv: "90f8924c64c589cf883974b0885662a9"
server_key_version: "encryptionkey@0"
I20260806 04:57:40.545475 14683 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785992227050689-9601-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785992227050689-9601-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785992227050689-9601-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 04:57:40.560186 14683 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 04:57:40.564253 14683 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 04:57:40.564568 14683 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 04:57:40.565474 14683 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 04:57:40.567101 14683 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 04:57:40.567190 14683 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 04:57:40.567315 14683 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 04:57:40.567436 14683 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.001s
I20260806 04:57:40.601598 14683 rpc_server.cc:307] RPC server started. Bound to: 127.9.96.67:44703
I20260806 04:57:40.601648 14879 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.96.67:44703 every 8 connection(s)
I20260806 04:57:40.602840 14683 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785992227050689-9601-0/minicluster-data/ts-2/data/info.pb
I20260806 04:57:40.609241 9601 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu as pid 14683
I20260806 04:57:40.609452 9601 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785992227050689-9601-0/minicluster-data/ts-2/wal/instance
I20260806 04:57:40.609928 9601 external_mini_cluster.cc:1428] Setting key 437547837dd8d1d2ad6a1c675ae922ba
I20260806 04:57:40.611007 14480 heartbeater.cc:503] Master 127.9.96.126:43727 was elected leader, sending a full tablet report...
Aug 06 04:57:40 dist-test-slave-x4qh krb5kdc[14122](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992260, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.96.67@KRBTEST.COM for kudu/127.9.96.126@KRBTEST.COM
I20260806 04:57:40.621978 14677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:57:40.605346 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.96.67:33223 (local address 127.9.96.126:43727)
0806 04:57:40.605522 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0806 04:57:40.605526 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 04:57:40.606473 (+ 947us) server_negotiation.cc:407] Connection header received
0806 04:57:40.608321 (+ 1848us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:57:40.608327 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:57:40.608421 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:57:40.608531 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:57:40.610027 (+ 1496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:57:40.611020 (+ 993us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:57:40.613270 (+ 2250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:57:40.613629 (+ 359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:57:40.613720 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:57:40.616896 (+ 3176us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:57:40.616901 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:57:40.616904 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:57:40.616935 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:57:40.619199 (+ 2264us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:57:40.619880 (+ 681us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:57:40.619883 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:57:40.619885 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:57:40.620021 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:57:40.620431 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:57:40.620434 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:57:40.620436 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:57:40.620685 (+ 249us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:57:40.620793 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0806 04:57:40.621481 (+ 688us) server_negotiation.cc:299] Negotiation successful
0806 04:57:40.621637 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260806 04:57:40.622817 14883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:57:40.605645 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:43727 (local address 127.9.96.67:33223)
0806 04:57:40.606314 (+ 669us) negotiation.cc:107] Waiting for socket to connect
0806 04:57:40.606357 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0806 04:57:40.608063 (+ 1706us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:57:40.608752 (+ 689us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:57:40.608760 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:57:40.609250 (+ 490us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:57:40.609859 (+ 609us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:57:40.609875 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:57:40.611214 (+ 1339us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:57:40.611220 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:57:40.613116 (+ 1896us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:57:40.613145 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:57:40.613800 (+ 655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:57:40.613804 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:57:40.613993 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:57:40.614570 (+ 577us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:57:40.614590 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:57:40.616710 (+ 2120us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 04:57:40.619317 (+ 2607us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:57:40.619323 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:57:40.619335 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:57:40.619768 (+ 433us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:57:40.620142 (+ 374us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:57:40.620145 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:57:40.620147 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:57:40.620324 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:57:40.620788 (+ 464us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 04:57:40.620793 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 04:57:40.621063 (+ 270us) client_negotiation.cc:769] Sending connection context
0806 04:57:40.621364 (+ 301us) client_negotiation.cc:240] Negotiation successful
0806 04:57:40.621719 (+ 355us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":455,"thread_start_us":272,"threads_started":1}
I20260806 04:57:40.624367 14880 heartbeater.cc:347] Connected to a master server at 127.9.96.126:43727
I20260806 04:57:40.624665 14880 heartbeater.cc:464] Registering TS with master...
I20260806 04:57:40.625352 14880 heartbeater.cc:511] Master 127.9.96.126:43727 requested a full tablet report, sending...
I20260806 04:57:40.626817 14188 ts_manager.cc:194] Registered new tserver with Master: 7a707dc40ea7417da7b5888478870cb9 (127.9.96.67:44703)
I20260806 04:57:40.627755 14188 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.96.67@KRBTEST.COM'} at 127.9.96.67:33223
I20260806 04:57:40.637390 9601 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 06 04:57:40 dist-test-slave-x4qh krb5kdc[14122](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992260, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Aug 06 04:57:40 dist-test-slave-x4qh krb5kdc[14122](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992260, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.9.96.126@KRBTEST.COM
I20260806 04:57:40.689793 14677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:57:40.675505 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59340 (local address 127.9.96.126:43727)
0806 04:57:40.675725 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0806 04:57:40.675734 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 04:57:40.676156 (+ 422us) server_negotiation.cc:407] Connection header received
0806 04:57:40.676384 (+ 228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:57:40.676395 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:57:40.676480 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:57:40.676595 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:57:40.677991 (+ 1396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:57:40.678733 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:57:40.680392 (+ 1659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:57:40.681061 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:57:40.681166 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:57:40.684805 (+ 3639us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:57:40.684810 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:57:40.684813 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:57:40.684854 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:57:40.687577 (+ 2723us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:57:40.688136 (+ 559us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:57:40.688140 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:57:40.688142 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:57:40.688283 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:57:40.688583 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:57:40.688586 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:57:40.688588 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:57:40.688891 (+ 303us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:57:40.689089 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0806 04:57:40.689300 (+ 211us) server_negotiation.cc:299] Negotiation successful
0806 04:57:40.689452 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
W20260806 04:57:40.690454 14188 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:59340
I20260806 04:57:40.692236 9601 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu pid 14294
Aug 06 04:57:40 dist-test-slave-x4qh krb5kdc[14122](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992260, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.9.96.65@KRBTEST.COM
I20260806 04:57:40.706702 14899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:57:40.693268 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59922 (local address 127.9.96.65:34695)
0806 04:57:40.693842 (+ 574us) server_negotiation.cc:206] Beginning negotiation
0806 04:57:40.693848 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 04:57:40.693872 (+ 24us) server_negotiation.cc:407] Connection header received
0806 04:57:40.693924 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:57:40.693930 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:57:40.694121 (+ 191us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:57:40.694236 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:57:40.695341 (+ 1105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:57:40.696335 (+ 994us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:57:40.697412 (+ 1077us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:57:40.697958 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:57:40.698046 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:57:40.701477 (+ 3431us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:57:40.701486 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:57:40.701499 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:57:40.701531 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:57:40.704330 (+ 2799us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:57:40.704977 (+ 647us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:57:40.704983 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:57:40.704985 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:57:40.705102 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:57:40.705401 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:57:40.705404 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:57:40.705406 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:57:40.705701 (+ 295us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:57:40.705922 (+ 221us) server_negotiation.cc:1035] Waiting for connection context
0806 04:57:40.706103 (+ 181us) server_negotiation.cc:299] Negotiation successful
0806 04:57:40.706278 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":424,"thread_start_us":295,"threads_started":1}
W20260806 04:57:40.708043 14475 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:59922
W20260806 04:57:40.709208 9601 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu pid 14294: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260806 04:57:40.709407 9601 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu with pid 14294
I20260806 04:57:40.734781 9601 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu pid 14490
Aug 06 04:57:40 dist-test-slave-x4qh krb5kdc[14122](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992260, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.9.96.66@KRBTEST.COM
I20260806 04:57:40.750268 14901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:57:40.735576 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52516 (local address 127.9.96.66:35995)
0806 04:57:40.736047 (+ 471us) server_negotiation.cc:206] Beginning negotiation
0806 04:57:40.736053 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 04:57:40.736076 (+ 23us) server_negotiation.cc:407] Connection header received
0806 04:57:40.736151 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:57:40.736157 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:57:40.736381 (+ 224us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:57:40.736505 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:57:40.737732 (+ 1227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:57:40.738956 (+ 1224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:57:40.740241 (+ 1285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:57:40.740740 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:57:40.740835 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:57:40.744751 (+ 3916us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:57:40.744762 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:57:40.744774 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:57:40.744817 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:57:40.747763 (+ 2946us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:57:40.748330 (+ 567us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:57:40.748336 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:57:40.748338 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:57:40.748456 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:57:40.748746 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:57:40.748749 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:57:40.748751 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:57:40.749123 (+ 372us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:57:40.749447 (+ 324us) server_negotiation.cc:1035] Waiting for connection context
0806 04:57:40.749642 (+ 195us) server_negotiation.cc:299] Negotiation successful
0806 04:57:40.749877 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":335,"thread_start_us":227,"threads_started":1}
W20260806 04:57:40.751545 14664 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:52516
W20260806 04:57:40.752549 9601 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu pid 14490: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260806 04:57:40.752709 9601 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu with pid 14490
I20260806 04:57:40.776191 9601 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu pid 14683
Aug 06 04:57:40 dist-test-slave-x4qh krb5kdc[14122](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992260, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.9.96.67@KRBTEST.COM
I20260806 04:57:40.792300 14903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:57:40.777155 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54642 (local address 127.9.96.67:44703)
0806 04:57:40.777812 (+ 657us) server_negotiation.cc:206] Beginning negotiation
0806 04:57:40.777819 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 04:57:40.777868 (+ 49us) server_negotiation.cc:407] Connection header received
0806 04:57:40.777923 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:57:40.777929 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:57:40.778123 (+ 194us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:57:40.778296 (+ 173us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:57:40.779661 (+ 1365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:57:40.780668 (+ 1007us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:57:40.781860 (+ 1192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:57:40.782434 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:57:40.782574 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:57:40.786819 (+ 4245us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:57:40.786835 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:57:40.786847 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:57:40.786879 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:57:40.789424 (+ 2545us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:57:40.790186 (+ 762us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:57:40.790193 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:57:40.790195 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:57:40.790313 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:57:40.790691 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:57:40.790697 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:57:40.790700 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:57:40.791030 (+ 330us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:57:40.791345 (+ 315us) server_negotiation.cc:1035] Waiting for connection context
0806 04:57:40.791696 (+ 351us) server_negotiation.cc:299] Negotiation successful
0806 04:57:40.791980 (+ 284us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":515,"thread_start_us":359,"threads_started":1}
W20260806 04:57:40.793573 14874 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:54642
W20260806 04:57:40.794600 9601 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu pid 14683: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260806 04:57:40.794759 9601 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu with pid 14683
I20260806 04:57:40.818235 9601 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu pid 14142
I20260806 04:57:40.819446 14244 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 04:57:40.922046 9601 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu with pid 14142
2026-08-06T04:57:40Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (3828 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785992227050689-9601-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 04:57:41 dist-test-slave-x4qh krb5kdc[14912](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 04:57:41 dist-test-slave-x4qh krb5kdc[14912](info): set up 2 sockets
Aug 06 04:57:41 dist-test-slave-x4qh krb5kdc[14912](info): commencing operation
krb5kdc: starting...
W20260806 04:57:42.201583 9601 mini_kdc.cc:121] Time spent starting KDC: real 1.230s user 0.001s sys 0.020s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785992227050689-9601-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785992227050689-9601-0/krb5kdc/test-user.keytab.
Aug 06 04:57:42 dist-test-slave-x4qh krb5kdc[14912](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992262, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T04:57:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T04:57:42Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.96.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.96.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.96.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.126.keytab.
Entry for principal kudu/127.9.96.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.126.keytab.
I20260806 04:57:42.403275 9601 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
/tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785992227050689-9601-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785992227050689-9601-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785992227050689-9601-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785992227050689-9601-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.96.126:34507
--webserver_interface=127.9.96.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.126.keytab
--principal=kudu/127.9.96.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.96.84:44019
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.96.126:34507
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785992227050689-9601-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785992227050689-9601-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785992227050689-9601-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 04:57:42.723222 14933 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 04:57:42.723590 14933 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 04:57:42.723685 14933 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 04:57:42.730822 14933 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 04:57:42.730913 14933 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 04:57:42.730957 14933 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 04:57:42.731011 14933 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 04:57:42.731063 14933 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 04:57:42.740911 14933 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.96.84:44019
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785992227050689-9601-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785992227050689-9601-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.96.126:34507
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.126.keytab
--principal=kudu/127.9.96.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.96.126:34507
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785992227050689-9601-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.96.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785992227050689-9601-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 03:56:12 UTC on ccycloud.quasar-guijbm.root.comops.site
build id 4569618
ASAN enabled
I20260806 04:57:42.742234 14933 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 04:57:42.743520 14933 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 04:57:42.755302 14941 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:57:42.755594 14943 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:57:42.755429 14946 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:57:42.756402 14933 server_base.cc:1034] running on GCE node
I20260806 04:57:42.757112 14933 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 04:57:42.758469 14933 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 04:57:42.759703 14933 hybrid_clock.cc:648] HybridClock initialized: now 1785992262759652 us; error 36 us; skew 500 ppm
I20260806 04:57:42.760038 14933 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 04:57:42 dist-test-slave-x4qh krb5kdc[14912](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992262, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.96.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 04:57:42.763151 14933 init.cc:377] Logged in from keytab as kudu/127.9.96.126@KRBTEST.COM (short username kudu)
I20260806 04:57:42.764726 14933 webserver.cc:460] Webserver started at http://127.9.96.126:42119/ using document root <none> and password file <none>
I20260806 04:57:42.765342 14933 fs_manager.cc:359] Metadata directory not provided
I20260806 04:57:42.765457 14933 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 04:57:42.765731 14933 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 04:57:42.768654 14933 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785992227050689-9601-0/minicluster-data/master-0/data/instance:
uuid: "9f2cb9d2c4314f6f9bf4080e9862321d"
format_stamp: "Formatted at 2026-08-06 04:57:42 on dist-test-slave-x4qh"
server_key: "5624cd16b775824da094088d3a0fe402"
server_key_iv: "d021eb2f3502be39dd10b1ed57b6976f"
server_key_version: "encryptionkey@0"
I20260806 04:57:42.769382 14933 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785992227050689-9601-0/minicluster-data/master-0/wal/instance:
uuid: "9f2cb9d2c4314f6f9bf4080e9862321d"
format_stamp: "Formatted at 2026-08-06 04:57:42 on dist-test-slave-x4qh"
server_key: "5624cd16b775824da094088d3a0fe402"
server_key_iv: "d021eb2f3502be39dd10b1ed57b6976f"
server_key_version: "encryptionkey@0"
I20260806 04:57:42.774232 14933 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260806 04:57:42.777567 14954 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 04:57:42.778887 14933 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260806 04:57:42.779028 14933 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785992227050689-9601-0/minicluster-data/master-0/data,/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785992227050689-9601-0/minicluster-data/master-0/wal
uuid: "9f2cb9d2c4314f6f9bf4080e9862321d"
format_stamp: "Formatted at 2026-08-06 04:57:42 on dist-test-slave-x4qh"
server_key: "5624cd16b775824da094088d3a0fe402"
server_key_iv: "d021eb2f3502be39dd10b1ed57b6976f"
server_key_version: "encryptionkey@0"
I20260806 04:57:42.779191 14933 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785992227050689-9601-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785992227050689-9601-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785992227050689-9601-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 04:57:42.807761 14933 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 04:57:42.812188 14933 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 04:57:42.812456 14933 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 04:57:42.831156 14933 rpc_server.cc:307] RPC server started. Bound to: 127.9.96.126:34507
I20260806 04:57:42.831197 15026 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.96.126:34507 every 8 connection(s)
I20260806 04:57:42.832584 14933 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785992227050689-9601-0/minicluster-data/master-0/data/info.pb
I20260806 04:57:42.836339 15027 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 04:57:42.841189 9601 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu as pid 14933
I20260806 04:57:42.841409 9601 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785992227050689-9601-0/minicluster-data/master-0/wal/instance
I20260806 04:57:42.842244 9601 external_mini_cluster.cc:1428] Setting key 7c0ee73c9d5fa8678abe22a71025ce28
I20260806 04:57:42.847810 15027 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9f2cb9d2c4314f6f9bf4080e9862321d: Bootstrap starting.
I20260806 04:57:42.851356 15027 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9f2cb9d2c4314f6f9bf4080e9862321d: Neither blocks nor log segments found. Creating new log.
I20260806 04:57:42.852591 15027 log.cc:824] T 00000000000000000000000000000000 P 9f2cb9d2c4314f6f9bf4080e9862321d: Log is configured to *not* fsync() on all Append() calls
I20260806 04:57:42.856091 15027 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9f2cb9d2c4314f6f9bf4080e9862321d: No bootstrap required, opened a new log
Aug 06 04:57:42 dist-test-slave-x4qh krb5kdc[14912](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992262, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.96.126@KRBTEST.COM
I20260806 04:57:42.863838 15027 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9f2cb9d2c4314f6f9bf4080e9862321d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f2cb9d2c4314f6f9bf4080e9862321d" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 34507 } }
I20260806 04:57:42.864224 15027 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9f2cb9d2c4314f6f9bf4080e9862321d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 04:57:42.864342 15027 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9f2cb9d2c4314f6f9bf4080e9862321d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9f2cb9d2c4314f6f9bf4080e9862321d, State: Initialized, Role: FOLLOWER
I20260806 04:57:42.865072 15027 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9f2cb9d2c4314f6f9bf4080e9862321d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f2cb9d2c4314f6f9bf4080e9862321d" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 34507 } }
I20260806 04:57:42.865343 15027 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9f2cb9d2c4314f6f9bf4080e9862321d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 04:57:42.865507 15027 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9f2cb9d2c4314f6f9bf4080e9862321d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 04:57:42.865715 15027 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9f2cb9d2c4314f6f9bf4080e9862321d [term 0 FOLLOWER]: Advancing to term 1
I20260806 04:57:42.866354 15030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:57:42.844392 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46188 (local address 127.9.96.126:34507)
0806 04:57:42.845262 (+ 870us) server_negotiation.cc:206] Beginning negotiation
0806 04:57:42.845276 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0806 04:57:42.845321 (+ 45us) server_negotiation.cc:407] Connection header received
0806 04:57:42.847048 (+ 1727us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:57:42.847070 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:57:42.847491 (+ 421us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:57:42.847791 (+ 300us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:57:42.849594 (+ 1803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:57:42.850844 (+ 1250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:57:42.852684 (+ 1840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:57:42.853721 (+ 1037us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:57:42.853930 (+ 209us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:57:42.858545 (+ 4615us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:57:42.858552 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:57:42.858568 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:57:42.858608 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:57:42.861877 (+ 3269us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:57:42.862628 (+ 751us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:57:42.862635 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:57:42.862641 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:57:42.862816 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:57:42.863251 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:57:42.863256 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:57:42.863258 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:57:42.863862 (+ 604us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:57:42.864163 (+ 301us) server_negotiation.cc:1035] Waiting for connection context
0806 04:57:42.864647 (+ 484us) server_negotiation.cc:299] Negotiation successful
0806 04:57:42.865107 (+ 460us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":631,"thread_start_us":420,"threads_started":1}
I20260806 04:57:42.868492 15027 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9f2cb9d2c4314f6f9bf4080e9862321d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f2cb9d2c4314f6f9bf4080e9862321d" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 34507 } }
I20260806 04:57:42.869061 15027 leader_election.cc:302] T 00000000000000000000000000000000 P 9f2cb9d2c4314f6f9bf4080e9862321d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9f2cb9d2c4314f6f9bf4080e9862321d; no voters:
I20260806 04:57:42.869827 15027 leader_election.cc:288] T 00000000000000000000000000000000 P 9f2cb9d2c4314f6f9bf4080e9862321d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 04:57:42.869920 15034 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9f2cb9d2c4314f6f9bf4080e9862321d [term 1 FOLLOWER]: Leader election won for term 1
I20260806 04:57:42.870693 15034 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9f2cb9d2c4314f6f9bf4080e9862321d [term 1 LEADER]: Becoming Leader. State: Replica: 9f2cb9d2c4314f6f9bf4080e9862321d, State: Running, Role: LEADER
I20260806 04:57:42.871229 15034 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9f2cb9d2c4314f6f9bf4080e9862321d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f2cb9d2c4314f6f9bf4080e9862321d" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 34507 } }
I20260806 04:57:42.872321 15027 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9f2cb9d2c4314f6f9bf4080e9862321d [sys.catalog]: configured and running, proceeding with master startup.
I20260806 04:57:42.874393 15037 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9f2cb9d2c4314f6f9bf4080e9862321d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9f2cb9d2c4314f6f9bf4080e9862321d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f2cb9d2c4314f6f9bf4080e9862321d" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 34507 } } }
I20260806 04:57:42.874897 15037 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9f2cb9d2c4314f6f9bf4080e9862321d [sys.catalog]: This master's current role is: LEADER
I20260806 04:57:42.875735 15039 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9f2cb9d2c4314f6f9bf4080e9862321d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9f2cb9d2c4314f6f9bf4080e9862321d. Latest consensus state: current_term: 1 leader_uuid: "9f2cb9d2c4314f6f9bf4080e9862321d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f2cb9d2c4314f6f9bf4080e9862321d" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 34507 } } }
I20260806 04:57:42.876104 15039 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9f2cb9d2c4314f6f9bf4080e9862321d [sys.catalog]: This master's current role is: LEADER
I20260806 04:57:42.877277 15051 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 04:57:42.882602 15051 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 04:57:42.891691 15051 catalog_manager.cc:1347] Generated new cluster ID: 26ba6a65444f4bc2a173dc0189cc65ae
I20260806 04:57:42.891805 15051 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 04:57:42.899333 15051 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 04:57:42.901108 15051 catalog_manager.cc:1504] Loading token signing keys...
I20260806 04:57:42.913910 15051 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9f2cb9d2c4314f6f9bf4080e9862321d: Generated new TSK 0
I20260806 04:57:42.914762 15051 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.9.96.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.96.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.96.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.65.keytab.
Entry for principal kudu/127.9.96.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.65.keytab.
I20260806 04:57:42.975154 9601 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
/tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785992227050689-9601-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785992227050689-9601-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785992227050689-9601-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785992227050689-9601-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.96.65:0
--local_ip_for_outbound_sockets=127.9.96.65
--webserver_interface=127.9.96.65
--webserver_port=0
--tserver_master_addrs=127.9.96.126:34507
--keytab_file=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.65.keytab
--principal=kudu/127.9.96.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.96.84:44019
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785992227050689-9601-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785992227050689-9601-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785992227050689-9601-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 04:57:43.302371 15069 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 04:57:43.302642 15069 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 04:57:43.302796 15069 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 04:57:43.309880 15069 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 04:57:43.309980 15069 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 04:57:43.310145 15069 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.96.65
I20260806 04:57:43.319409 15069 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.96.84:44019
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785992227050689-9601-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785992227050689-9601-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.65.keytab
--principal=kudu/127.9.96.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.96.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785992227050689-9601-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.96.65
--webserver_port=0
--tserver_master_addrs=127.9.96.126:34507
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.96.65
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785992227050689-9601-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 03:56:12 UTC on ccycloud.quasar-guijbm.root.comops.site
build id 4569618
ASAN enabled
I20260806 04:57:43.320638 15069 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 04:57:43.321949 15069 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 04:57:43.335175 15077 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:57:43.335175 15081 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:57:43.335208 15083 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:57:43.336170 15069 server_base.cc:1034] running on GCE node
I20260806 04:57:43.336849 15069 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 04:57:43.337787 15069 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 04:57:43.339036 15069 hybrid_clock.cc:648] HybridClock initialized: now 1785992263339001 us; error 49 us; skew 500 ppm
I20260806 04:57:43.339412 15069 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 04:57:43 dist-test-slave-x4qh krb5kdc[14912](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992263, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.96.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 04:57:43.342546 15069 init.cc:377] Logged in from keytab as kudu/127.9.96.65@KRBTEST.COM (short username kudu)
I20260806 04:57:43.344064 15069 webserver.cc:460] Webserver started at http://127.9.96.65:37903/ using document root <none> and password file <none>
I20260806 04:57:43.344774 15069 fs_manager.cc:359] Metadata directory not provided
I20260806 04:57:43.344926 15069 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 04:57:43.345247 15069 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 04:57:43.348219 15069 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785992227050689-9601-0/minicluster-data/ts-0/data/instance:
uuid: "a5be5a9386e04b738c062b9e7d5d31f4"
format_stamp: "Formatted at 2026-08-06 04:57:43 on dist-test-slave-x4qh"
server_key: "ed510c63ab267f244a2a62c8bd6d07b9"
server_key_iv: "6fa1051e8ef7fbfdd171c4bc401a6be1"
server_key_version: "encryptionkey@0"
I20260806 04:57:43.348995 15069 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785992227050689-9601-0/minicluster-data/ts-0/wal/instance:
uuid: "a5be5a9386e04b738c062b9e7d5d31f4"
format_stamp: "Formatted at 2026-08-06 04:57:43 on dist-test-slave-x4qh"
server_key: "ed510c63ab267f244a2a62c8bd6d07b9"
server_key_iv: "6fa1051e8ef7fbfdd171c4bc401a6be1"
server_key_version: "encryptionkey@0"
I20260806 04:57:43.354174 15069 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260806 04:57:43.357527 15091 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 04:57:43.358789 15069 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260806 04:57:43.358965 15069 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785992227050689-9601-0/minicluster-data/ts-0/data,/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785992227050689-9601-0/minicluster-data/ts-0/wal
uuid: "a5be5a9386e04b738c062b9e7d5d31f4"
format_stamp: "Formatted at 2026-08-06 04:57:43 on dist-test-slave-x4qh"
server_key: "ed510c63ab267f244a2a62c8bd6d07b9"
server_key_iv: "6fa1051e8ef7fbfdd171c4bc401a6be1"
server_key_version: "encryptionkey@0"
I20260806 04:57:43.359164 15069 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785992227050689-9601-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785992227050689-9601-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785992227050689-9601-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 04:57:43.382242 15069 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 04:57:43.386360 15069 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 04:57:43.386674 15069 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 04:57:43.387679 15069 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 04:57:43.389014 15069 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 04:57:43.389123 15069 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 04:57:43.389278 15069 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 04:57:43.389353 15069 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 04:57:43.424185 15069 rpc_server.cc:307] RPC server started. Bound to: 127.9.96.65:34489
I20260806 04:57:43.424309 15278 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.96.65:34489 every 8 connection(s)
I20260806 04:57:43.425498 15069 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785992227050689-9601-0/minicluster-data/ts-0/data/info.pb
I20260806 04:57:43.434943 9601 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu as pid 15069
I20260806 04:57:43.435145 9601 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785992227050689-9601-0/minicluster-data/ts-0/wal/instance
I20260806 04:57:43.435928 9601 external_mini_cluster.cc:1428] Setting key c77b2649810c550e600048e297472d93
Aug 06 04:57:43 dist-test-slave-x4qh krb5kdc[14912](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992263, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.96.65@KRBTEST.COM for kudu/127.9.96.126@KRBTEST.COM
I20260806 04:57:43.445226 15284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:57:43.427978 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.96.65:42195 (local address 127.9.96.126:34507)
0806 04:57:43.428435 (+ 457us) server_negotiation.cc:206] Beginning negotiation
0806 04:57:43.428439 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 04:57:43.429289 (+ 850us) server_negotiation.cc:407] Connection header received
0806 04:57:43.431087 (+ 1798us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:57:43.431095 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:57:43.431188 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:57:43.431304 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:57:43.432737 (+ 1433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:57:43.433433 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:57:43.435490 (+ 2057us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:57:43.435903 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:57:43.436042 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:57:43.439704 (+ 3662us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:57:43.439710 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:57:43.439715 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:57:43.439752 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:57:43.442189 (+ 2437us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:57:43.442922 (+ 733us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:57:43.442927 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:57:43.442931 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:57:43.443054 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:57:43.443527 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:57:43.443532 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:57:43.443536 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:57:43.443790 (+ 254us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:57:43.443949 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0806 04:57:43.444649 (+ 700us) server_negotiation.cc:299] Negotiation successful
0806 04:57:43.444872 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":354,"thread_start_us":264,"threads_started":1}
I20260806 04:57:43.446147 15285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:57:43.428443 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:34507 (local address 127.9.96.65:42195)
0806 04:57:43.429120 (+ 677us) negotiation.cc:107] Waiting for socket to connect
0806 04:57:43.429162 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0806 04:57:43.430824 (+ 1662us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:57:43.431545 (+ 721us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:57:43.431552 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:57:43.432030 (+ 478us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:57:43.432560 (+ 530us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:57:43.432575 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:57:43.433644 (+ 1069us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:57:43.433648 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:57:43.435276 (+ 1628us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:57:43.435290 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:57:43.436166 (+ 876us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:57:43.436183 (+ 17us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:57:43.436449 (+ 266us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:57:43.437190 (+ 741us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:57:43.437212 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:57:43.439512 (+ 2300us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 04:57:43.442335 (+ 2823us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:57:43.442343 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:57:43.442357 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:57:43.442765 (+ 408us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:57:43.443167 (+ 402us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:57:43.443172 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:57:43.443176 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:57:43.443360 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:57:43.443927 (+ 567us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 04:57:43.443934 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 04:57:43.444247 (+ 313us) client_negotiation.cc:769] Sending connection context
0806 04:57:43.444523 (+ 276us) client_negotiation.cc:240] Negotiation successful
0806 04:57:43.445025 (+ 502us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":470,"thread_start_us":274,"threads_started":1}
I20260806 04:57:43.447726 15279 heartbeater.cc:347] Connected to a master server at 127.9.96.126:34507
I20260806 04:57:43.448048 15279 heartbeater.cc:464] Registering TS with master...
I20260806 04:57:43.448932 15279 heartbeater.cc:511] Master 127.9.96.126:34507 requested a full tablet report, sending...
I20260806 04:57:43.451316 14981 ts_manager.cc:194] Registered new tserver with Master: a5be5a9386e04b738c062b9e7d5d31f4 (127.9.96.65:34489)
I20260806 04:57:43.453517 14981 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.96.65@KRBTEST.COM'} at 127.9.96.65:42195
WARNING: no policy specified for kudu/127.9.96.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.96.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.96.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.66.keytab.
Entry for principal kudu/127.9.96.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.66.keytab.
I20260806 04:57:43.485910 9601 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
/tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785992227050689-9601-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785992227050689-9601-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785992227050689-9601-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785992227050689-9601-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.96.66:0
--local_ip_for_outbound_sockets=127.9.96.66
--webserver_interface=127.9.96.66
--webserver_port=0
--tserver_master_addrs=127.9.96.126:34507
--keytab_file=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.66.keytab
--principal=kudu/127.9.96.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.96.84:44019
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785992227050689-9601-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785992227050689-9601-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785992227050689-9601-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 04:57:43.805253 15288 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 04:57:43.805509 15288 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 04:57:43.805600 15288 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 04:57:43.812521 15288 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 04:57:43.812618 15288 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 04:57:43.812722 15288 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.96.66
I20260806 04:57:43.821883 15288 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.96.84:44019
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785992227050689-9601-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785992227050689-9601-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.66.keytab
--principal=kudu/127.9.96.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.96.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785992227050689-9601-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.96.66
--webserver_port=0
--tserver_master_addrs=127.9.96.126:34507
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.96.66
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785992227050689-9601-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 03:56:12 UTC on ccycloud.quasar-guijbm.root.comops.site
build id 4569618
ASAN enabled
I20260806 04:57:43.823012 15288 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 04:57:43.824326 15288 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 04:57:43.837368 15298 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:57:43.837565 15295 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:57:43.837836 15300 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:57:43.838835 15288 server_base.cc:1034] running on GCE node
I20260806 04:57:43.839529 15288 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 04:57:43.840480 15288 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 04:57:43.841701 15288 hybrid_clock.cc:648] HybridClock initialized: now 1785992263841652 us; error 37 us; skew 500 ppm
I20260806 04:57:43.842029 15288 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 04:57:43 dist-test-slave-x4qh krb5kdc[14912](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992263, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.96.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 04:57:43.845705 15288 init.cc:377] Logged in from keytab as kudu/127.9.96.66@KRBTEST.COM (short username kudu)
I20260806 04:57:43.850126 15288 webserver.cc:460] Webserver started at http://127.9.96.66:34813/ using document root <none> and password file <none>
I20260806 04:57:43.850837 15288 fs_manager.cc:359] Metadata directory not provided
I20260806 04:57:43.850991 15288 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 04:57:43.851312 15288 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 04:57:43.854137 15288 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785992227050689-9601-0/minicluster-data/ts-1/data/instance:
uuid: "089f84b6d0254a73b7cea8be22246b1a"
format_stamp: "Formatted at 2026-08-06 04:57:43 on dist-test-slave-x4qh"
server_key: "2d4ae45d39976be52df88e5559dbc364"
server_key_iv: "e62fd79de834c27a35c371dc4bdb4507"
server_key_version: "encryptionkey@0"
I20260806 04:57:43.854923 15288 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785992227050689-9601-0/minicluster-data/ts-1/wal/instance:
uuid: "089f84b6d0254a73b7cea8be22246b1a"
format_stamp: "Formatted at 2026-08-06 04:57:43 on dist-test-slave-x4qh"
server_key: "2d4ae45d39976be52df88e5559dbc364"
server_key_iv: "e62fd79de834c27a35c371dc4bdb4507"
server_key_version: "encryptionkey@0"
I20260806 04:57:43.860173 15288 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.002s
I20260806 04:57:43.863620 15314 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 04:57:43.864908 15288 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260806 04:57:43.865083 15288 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785992227050689-9601-0/minicluster-data/ts-1/data,/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785992227050689-9601-0/minicluster-data/ts-1/wal
uuid: "089f84b6d0254a73b7cea8be22246b1a"
format_stamp: "Formatted at 2026-08-06 04:57:43 on dist-test-slave-x4qh"
server_key: "2d4ae45d39976be52df88e5559dbc364"
server_key_iv: "e62fd79de834c27a35c371dc4bdb4507"
server_key_version: "encryptionkey@0"
I20260806 04:57:43.865281 15288 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785992227050689-9601-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785992227050689-9601-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785992227050689-9601-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 04:57:43.890525 15288 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 04:57:43.895267 15288 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 04:57:43.895612 15288 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 04:57:43.896593 15288 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 04:57:43.897945 15288 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 04:57:43.898052 15288 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 04:57:43.898207 15288 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 04:57:43.898283 15288 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 04:57:43.931870 15288 rpc_server.cc:307] RPC server started. Bound to: 127.9.96.66:45351
I20260806 04:57:43.931926 15488 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.96.66:45351 every 8 connection(s)
I20260806 04:57:43.933184 15288 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785992227050689-9601-0/minicluster-data/ts-1/data/info.pb
I20260806 04:57:43.934245 9601 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu as pid 15288
I20260806 04:57:43.934437 9601 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785992227050689-9601-0/minicluster-data/ts-1/wal/instance
I20260806 04:57:43.935006 9601 external_mini_cluster.cc:1428] Setting key 0760ce7713bd41cf07d2a47f73f1e94e
Aug 06 04:57:43 dist-test-slave-x4qh krb5kdc[14912](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992263, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.96.66@KRBTEST.COM for kudu/127.9.96.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.9.96.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.96.67@KRBTEST.COM" created.
I20260806 04:57:43.961174 15284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:57:43.936160 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.96.66:41225 (local address 127.9.96.126:34507)
0806 04:57:43.936959 (+ 799us) server_negotiation.cc:206] Beginning negotiation
0806 04:57:43.936968 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 04:57:43.937674 (+ 706us) server_negotiation.cc:407] Connection header received
0806 04:57:43.940545 (+ 2871us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:57:43.940549 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:57:43.940614 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:57:43.940698 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:57:43.942757 (+ 2059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:57:43.943524 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:57:43.945800 (+ 2276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:57:43.946493 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:57:43.946634 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:57:43.954578 (+ 7944us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:57:43.954586 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:57:43.954593 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:57:43.954642 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:57:43.958120 (+ 3478us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:57:43.958796 (+ 676us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:57:43.958800 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:57:43.958802 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:57:43.958957 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:57:43.959447 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:57:43.959450 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:57:43.959452 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:57:43.959701 (+ 249us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:57:43.959812 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0806 04:57:43.960670 (+ 858us) server_negotiation.cc:299] Negotiation successful
0806 04:57:43.960925 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":662}
I20260806 04:57:43.962448 15492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:57:43.936634 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:34507 (local address 127.9.96.66:41225)
0806 04:57:43.937500 (+ 866us) negotiation.cc:107] Waiting for socket to connect
0806 04:57:43.937554 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0806 04:57:43.940245 (+ 2691us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:57:43.940963 (+ 718us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:57:43.940976 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:57:43.941773 (+ 797us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:57:43.942575 (+ 802us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:57:43.942601 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:57:43.943664 (+ 1063us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:57:43.943671 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:57:43.945664 (+ 1993us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:57:43.945675 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:57:43.946678 (+ 1003us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:57:43.946683 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:57:43.946864 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:57:43.951876 (+ 5012us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:57:43.951899 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:57:43.954328 (+ 2429us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 04:57:43.958253 (+ 3925us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:57:43.958259 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:57:43.958271 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:57:43.958689 (+ 418us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:57:43.959089 (+ 400us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:57:43.959097 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:57:43.959103 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:57:43.959294 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:57:43.959815 (+ 521us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 04:57:43.959821 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 04:57:43.960251 (+ 430us) client_negotiation.cc:769] Sending connection context
0806 04:57:43.960560 (+ 309us) client_negotiation.cc:240] Negotiation successful
0806 04:57:43.961014 (+ 454us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":590,"thread_start_us":349,"threads_started":1}
I20260806 04:57:43.963925 15489 heartbeater.cc:347] Connected to a master server at 127.9.96.126:34507
I20260806 04:57:43.964241 15489 heartbeater.cc:464] Registering TS with master...
I20260806 04:57:43.964962 15489 heartbeater.cc:511] Master 127.9.96.126:34507 requested a full tablet report, sending...
I20260806 04:57:43.966392 14981 ts_manager.cc:194] Registered new tserver with Master: 089f84b6d0254a73b7cea8be22246b1a (127.9.96.66:45351)
I20260806 04:57:43.967675 14981 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.96.66@KRBTEST.COM'} at 127.9.96.66:41225
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.96.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.67.keytab.
Entry for principal kudu/127.9.96.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.67.keytab.
I20260806 04:57:43.986603 9601 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
/tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785992227050689-9601-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785992227050689-9601-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785992227050689-9601-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785992227050689-9601-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.96.67:0
--local_ip_for_outbound_sockets=127.9.96.67
--webserver_interface=127.9.96.67
--webserver_port=0
--tserver_master_addrs=127.9.96.126:34507
--keytab_file=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.67.keytab
--principal=kudu/127.9.96.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.96.84:44019
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785992227050689-9601-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785992227050689-9601-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785992227050689-9601-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 04:57:44.306595 15495 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 04:57:44.306864 15495 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 04:57:44.306959 15495 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 04:57:44.313989 15495 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 04:57:44.314075 15495 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 04:57:44.314180 15495 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.96.67
I20260806 04:57:44.323201 15495 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.96.84:44019
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785992227050689-9601-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785992227050689-9601-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.67.keytab
--principal=kudu/127.9.96.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.96.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785992227050689-9601-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.96.67
--webserver_port=0
--tserver_master_addrs=127.9.96.126:34507
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.96.67
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785992227050689-9601-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 03:56:12 UTC on ccycloud.quasar-guijbm.root.comops.site
build id 4569618
ASAN enabled
I20260806 04:57:44.324441 15495 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 04:57:44.325670 15495 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 04:57:44.338265 15500 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:57:44.338188 15503 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:57:44.338233 15501 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:57:44.339270 15495 server_base.cc:1034] running on GCE node
I20260806 04:57:44.339880 15495 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 04:57:44.340701 15495 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 04:57:44.341894 15495 hybrid_clock.cc:648] HybridClock initialized: now 1785992264341850 us; error 36 us; skew 500 ppm
I20260806 04:57:44.342150 15495 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 04:57:44 dist-test-slave-x4qh krb5kdc[14912](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992264, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.96.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 04:57:44.345315 15495 init.cc:377] Logged in from keytab as kudu/127.9.96.67@KRBTEST.COM (short username kudu)
I20260806 04:57:44.346642 15495 webserver.cc:460] Webserver started at http://127.9.96.67:41643/ using document root <none> and password file <none>
I20260806 04:57:44.347254 15495 fs_manager.cc:359] Metadata directory not provided
I20260806 04:57:44.347361 15495 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 04:57:44.347733 15495 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 04:57:44.350466 15495 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785992227050689-9601-0/minicluster-data/ts-2/data/instance:
uuid: "7688e2f88679468fb85afc81b4ea7f5c"
format_stamp: "Formatted at 2026-08-06 04:57:44 on dist-test-slave-x4qh"
server_key: "a60bd09e7b9ba86934a329ffa5e78558"
server_key_iv: "18008fecb5ccb314617b4eb32d919a43"
server_key_version: "encryptionkey@0"
I20260806 04:57:44.351231 15495 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785992227050689-9601-0/minicluster-data/ts-2/wal/instance:
uuid: "7688e2f88679468fb85afc81b4ea7f5c"
format_stamp: "Formatted at 2026-08-06 04:57:44 on dist-test-slave-x4qh"
server_key: "a60bd09e7b9ba86934a329ffa5e78558"
server_key_iv: "18008fecb5ccb314617b4eb32d919a43"
server_key_version: "encryptionkey@0"
I20260806 04:57:44.356150 15495 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.001s
I20260806 04:57:44.359298 15513 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 04:57:44.360541 15495 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260806 04:57:44.360733 15495 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785992227050689-9601-0/minicluster-data/ts-2/data,/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785992227050689-9601-0/minicluster-data/ts-2/wal
uuid: "7688e2f88679468fb85afc81b4ea7f5c"
format_stamp: "Formatted at 2026-08-06 04:57:44 on dist-test-slave-x4qh"
server_key: "a60bd09e7b9ba86934a329ffa5e78558"
server_key_iv: "18008fecb5ccb314617b4eb32d919a43"
server_key_version: "encryptionkey@0"
I20260806 04:57:44.360886 15495 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785992227050689-9601-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785992227050689-9601-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785992227050689-9601-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 04:57:44.380641 15495 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 04:57:44.384646 15495 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 04:57:44.384943 15495 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 04:57:44.385818 15495 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 04:57:44.387092 15495 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 04:57:44.387194 15495 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 04:57:44.387348 15495 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 04:57:44.387458 15495 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.001s sys 0.000s
I20260806 04:57:44.419643 15495 rpc_server.cc:307] RPC server started. Bound to: 127.9.96.67:43919
I20260806 04:57:44.419687 15688 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.96.67:43919 every 8 connection(s)
I20260806 04:57:44.420768 15495 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785992227050689-9601-0/minicluster-data/ts-2/data/info.pb
I20260806 04:57:44.425802 9601 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu as pid 15495
I20260806 04:57:44.425973 9601 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785992227050689-9601-0/minicluster-data/ts-2/wal/instance
I20260806 04:57:44.426492 9601 external_mini_cluster.cc:1428] Setting key 8c21fab451b182431e8903d58fcdaf72
Aug 06 04:57:44 dist-test-slave-x4qh krb5kdc[14912](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992264, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.96.67@KRBTEST.COM for kudu/127.9.96.126@KRBTEST.COM
I20260806 04:57:44.441574 15284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:57:44.423176 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.96.67:58913 (local address 127.9.96.126:34507)
0806 04:57:44.423444 (+ 268us) server_negotiation.cc:206] Beginning negotiation
0806 04:57:44.423449 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 04:57:44.425258 (+ 1809us) server_negotiation.cc:407] Connection header received
0806 04:57:44.427840 (+ 2582us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:57:44.427847 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:57:44.427929 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:57:44.428031 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:57:44.429899 (+ 1868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:57:44.430606 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:57:44.432236 (+ 1630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:57:44.432641 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:57:44.432736 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:57:44.435909 (+ 3173us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:57:44.435913 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:57:44.435916 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:57:44.435947 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:57:44.438588 (+ 2641us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:57:44.439221 (+ 633us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:57:44.439224 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:57:44.439226 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:57:44.439355 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:57:44.439758 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:57:44.439761 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:57:44.439763 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:57:44.440009 (+ 246us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:57:44.440137 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0806 04:57:44.441020 (+ 883us) server_negotiation.cc:299] Negotiation successful
0806 04:57:44.441220 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260806 04:57:44.442363 15692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:57:44.423825 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:34507 (local address 127.9.96.67:58913)
0806 04:57:44.425022 (+ 1197us) negotiation.cc:107] Waiting for socket to connect
0806 04:57:44.425102 (+ 80us) client_negotiation.cc:174] Beginning negotiation
0806 04:57:44.427511 (+ 2409us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:57:44.428271 (+ 760us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:57:44.428278 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:57:44.428889 (+ 611us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:57:44.429730 (+ 841us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:57:44.429752 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:57:44.430730 (+ 978us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:57:44.430734 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:57:44.432000 (+ 1266us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:57:44.432011 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:57:44.432827 (+ 816us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:57:44.432842 (+ 15us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:57:44.433030 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:57:44.433593 (+ 563us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:57:44.433619 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:57:44.435703 (+ 2084us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 04:57:44.438699 (+ 2996us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:57:44.438704 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:57:44.438715 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:57:44.439104 (+ 389us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:57:44.439497 (+ 393us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:57:44.439501 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:57:44.439503 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:57:44.439662 (+ 159us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:57:44.440155 (+ 493us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 04:57:44.440162 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 04:57:44.440492 (+ 330us) client_negotiation.cc:769] Sending connection context
0806 04:57:44.440738 (+ 246us) client_negotiation.cc:240] Negotiation successful
0806 04:57:44.441209 (+ 471us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":776,"thread_start_us":455,"threads_started":1}
I20260806 04:57:44.443580 15690 heartbeater.cc:347] Connected to a master server at 127.9.96.126:34507
I20260806 04:57:44.443909 15690 heartbeater.cc:464] Registering TS with master...
I20260806 04:57:44.444602 15690 heartbeater.cc:511] Master 127.9.96.126:34507 requested a full tablet report, sending...
I20260806 04:57:44.445912 14981 ts_manager.cc:194] Registered new tserver with Master: 7688e2f88679468fb85afc81b4ea7f5c (127.9.96.67:43919)
I20260806 04:57:44.446677 14981 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.96.67@KRBTEST.COM'} at 127.9.96.67:58913
I20260806 04:57:44.454352 9601 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 04:57:44.457270 15279 heartbeater.cc:503] Master 127.9.96.126:34507 was elected leader, sending a full tablet report...
Aug 06 04:57:44 dist-test-slave-x4qh krb5kdc[14912](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992264, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 06 04:57:44 dist-test-slave-x4qh krb5kdc[14912](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992264, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.96.65@KRBTEST.COM
I20260806 04:57:44.506209 15700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:57:44.491155 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44536 (local address 127.9.96.65:34489)
0806 04:57:44.491807 (+ 652us) server_negotiation.cc:206] Beginning negotiation
0806 04:57:44.491814 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 04:57:44.491841 (+ 27us) server_negotiation.cc:407] Connection header received
0806 04:57:44.491907 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:57:44.491913 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:57:44.492111 (+ 198us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:57:44.492253 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:57:44.493856 (+ 1603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:57:44.494864 (+ 1008us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:57:44.496109 (+ 1245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:57:44.496658 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:57:44.496758 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:57:44.500915 (+ 4157us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:57:44.500925 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:57:44.500938 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:57:44.500965 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:57:44.503668 (+ 2703us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:57:44.504264 (+ 596us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:57:44.504270 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:57:44.504272 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:57:44.504407 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:57:44.504769 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:57:44.504772 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:57:44.504774 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:57:44.505079 (+ 305us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:57:44.505293 (+ 214us) server_negotiation.cc:1035] Waiting for connection context
0806 04:57:44.505613 (+ 320us) server_negotiation.cc:299] Negotiation successful
0806 04:57:44.505836 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":506,"thread_start_us":367,"threads_started":1}
Aug 06 04:57:44 dist-test-slave-x4qh krb5kdc[14912](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992264, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.96.126@KRBTEST.COM
I20260806 04:57:44.522493 15284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:57:44.510409 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46204 (local address 127.9.96.126:34507)
0806 04:57:44.510630 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0806 04:57:44.510638 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 04:57:44.510731 (+ 93us) server_negotiation.cc:407] Connection header received
0806 04:57:44.510849 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:57:44.510853 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:57:44.510921 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:57:44.510999 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:57:44.512264 (+ 1265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:57:44.512950 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:57:44.514024 (+ 1074us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:57:44.514402 (+ 378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:57:44.514489 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:57:44.517954 (+ 3465us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:57:44.517959 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:57:44.517962 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:57:44.517992 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:57:44.520335 (+ 2343us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:57:44.520897 (+ 562us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:57:44.520901 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:57:44.520903 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:57:44.521018 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:57:44.521310 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:57:44.521312 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:57:44.521314 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:57:44.521581 (+ 267us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:57:44.521754 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0806 04:57:44.522026 (+ 272us) server_negotiation.cc:299] Negotiation successful
0806 04:57:44.522169 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
W20260806 04:57:44.523048 14981 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46204
I20260806 04:57:44.524044 9601 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu pid 15069
I20260806 04:57:44.535477 15700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:57:44.524827 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44544 (local address 127.9.96.65:34489)
0806 04:57:44.524982 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0806 04:57:44.524985 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 04:57:44.525319 (+ 334us) server_negotiation.cc:407] Connection header received
0806 04:57:44.525448 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:57:44.525451 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:57:44.525533 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:57:44.525611 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:57:44.526711 (+ 1100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:57:44.527337 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:57:44.528474 (+ 1137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:57:44.528841 (+ 367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:57:44.528953 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:57:44.531059 (+ 2106us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:57:44.531063 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:57:44.531066 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:57:44.531095 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:57:44.533496 (+ 2401us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:57:44.534043 (+ 547us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:57:44.534046 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:57:44.534048 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:57:44.534154 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:57:44.534450 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:57:44.534453 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:57:44.534455 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:57:44.534671 (+ 216us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:57:44.534832 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0806 04:57:44.535060 (+ 228us) server_negotiation.cc:299] Negotiation successful
0806 04:57:44.535201 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260806 04:57:44.536010 15272 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 04:57:44.616570 9601 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu with pid 15069
I20260806 04:57:44.639041 9601 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu pid 15288
Aug 06 04:57:44 dist-test-slave-x4qh krb5kdc[14912](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992264, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.96.66@KRBTEST.COM
I20260806 04:57:44.654222 15706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:57:44.639984 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37760 (local address 127.9.96.66:45351)
0806 04:57:44.640583 (+ 599us) server_negotiation.cc:206] Beginning negotiation
0806 04:57:44.640590 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 04:57:44.640615 (+ 25us) server_negotiation.cc:407] Connection header received
0806 04:57:44.640678 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:57:44.640684 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:57:44.640891 (+ 207us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:57:44.641021 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:57:44.642141 (+ 1120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:57:44.643147 (+ 1006us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:57:44.644440 (+ 1293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:57:44.644999 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:57:44.645108 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:57:44.648998 (+ 3890us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:57:44.649010 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:57:44.649024 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:57:44.649055 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:57:44.651979 (+ 2924us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:57:44.652597 (+ 618us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:57:44.652603 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:57:44.652605 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:57:44.652722 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:57:44.653034 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:57:44.653037 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:57:44.653039 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:57:44.653354 (+ 315us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:57:44.653576 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0806 04:57:44.653761 (+ 185us) server_negotiation.cc:299] Negotiation successful
0806 04:57:44.653940 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":455,"thread_start_us":291,"threads_started":1}
I20260806 04:57:44.655865 15481 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 04:57:44.738308 9601 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu with pid 15288
I20260806 04:57:44.765084 9601 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu pid 15495
Aug 06 04:57:44 dist-test-slave-x4qh krb5kdc[14912](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992264, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.96.67@KRBTEST.COM
I20260806 04:57:44.781134 15710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:57:44.766217 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50482 (local address 127.9.96.67:43919)
0806 04:57:44.766885 (+ 668us) server_negotiation.cc:206] Beginning negotiation
0806 04:57:44.766891 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 04:57:44.766920 (+ 29us) server_negotiation.cc:407] Connection header received
0806 04:57:44.766981 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:57:44.766988 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:57:44.767209 (+ 221us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:57:44.767339 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:57:44.768577 (+ 1238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:57:44.769579 (+ 1002us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:57:44.770819 (+ 1240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:57:44.771532 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:57:44.771629 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:57:44.775895 (+ 4266us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:57:44.775905 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:57:44.775919 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:57:44.775953 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:57:44.778644 (+ 2691us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:57:44.779218 (+ 574us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:57:44.779225 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:57:44.779227 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:57:44.779358 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:57:44.779772 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:57:44.779775 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:57:44.779777 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:57:44.780091 (+ 314us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:57:44.780339 (+ 248us) server_negotiation.cc:1035] Waiting for connection context
0806 04:57:44.780566 (+ 227us) server_negotiation.cc:299] Negotiation successful
0806 04:57:44.780780 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":520,"thread_start_us":353,"threads_started":1}
I20260806 04:57:44.782517 15681 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 04:57:44.865270 9601 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu with pid 15495
I20260806 04:57:44.888501 9601 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu pid 14933
I20260806 04:57:44.889670 15021 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 04:57:44.989778 9601 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu with pid 14933
2026-08-06T04:57:45Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (4067 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785992227050689-9601-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 04:57:45 dist-test-slave-x4qh krb5kdc[15718](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 04:57:45 dist-test-slave-x4qh krb5kdc[15718](info): set up 2 sockets
Aug 06 04:57:45 dist-test-slave-x4qh krb5kdc[15718](info): commencing operation
krb5kdc: starting...
W20260806 04:57:46.276678 9601 mini_kdc.cc:121] Time spent starting KDC: real 1.237s user 0.003s sys 0.019s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785992227050689-9601-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785992227050689-9601-0/krb5kdc/test-user.keytab.
Aug 06 04:57:46 dist-test-slave-x4qh krb5kdc[15718](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992266, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T04:57:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T04:57:46Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.96.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.96.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.96.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.126.keytab.
Entry for principal kudu/127.9.96.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.126.keytab.
I20260806 04:57:46.478945 9601 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
/tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785992227050689-9601-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785992227050689-9601-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785992227050689-9601-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785992227050689-9601-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.96.126:36955
--webserver_interface=127.9.96.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.126.keytab
--principal=kudu/127.9.96.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.96.84:39773
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.96.126:36955
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785992227050689-9601-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785992227050689-9601-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785992227050689-9601-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 04:57:46.794032 15739 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 04:57:46.794368 15739 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 04:57:46.794549 15739 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 04:57:46.802214 15739 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 04:57:46.802376 15739 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 04:57:46.802488 15739 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 04:57:46.802591 15739 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 04:57:46.802672 15739 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 04:57:46.812743 15739 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.96.84:39773
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785992227050689-9601-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785992227050689-9601-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.96.126:36955
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.126.keytab
--principal=kudu/127.9.96.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.96.126:36955
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785992227050689-9601-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.96.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785992227050689-9601-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 03:56:12 UTC on ccycloud.quasar-guijbm.root.comops.site
build id 4569618
ASAN enabled
I20260806 04:57:46.814050 15739 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 04:57:46.815333 15739 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 04:57:46.826905 15758 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:57:46.826887 15756 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:57:46.827055 15753 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:57:46.828079 15739 server_base.cc:1034] running on GCE node
I20260806 04:57:46.828853 15739 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 04:57:46.830190 15739 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 04:57:46.831463 15739 hybrid_clock.cc:648] HybridClock initialized: now 1785992266831431 us; error 48 us; skew 500 ppm
I20260806 04:57:46.831818 15739 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 04:57:46 dist-test-slave-x4qh krb5kdc[15718](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992266, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.96.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 04:57:46.835073 15739 init.cc:377] Logged in from keytab as kudu/127.9.96.126@KRBTEST.COM (short username kudu)
I20260806 04:57:46.836261 15739 fs_manager.cc:359] Metadata directory not provided
I20260806 04:57:46.836382 15739 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 04:57:46.836668 15739 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 04:57:46.839706 15739 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785992227050689-9601-0/minicluster-data/master-0/data/instance:
uuid: "3f6af880361b42bfb3fdfe2fd232c604"
format_stamp: "Formatted at 2026-08-06 04:57:46 on dist-test-slave-x4qh"
server_key: "6bef4d5844b65bdb1daba3dd664b5007"
server_key_iv: "63ffa0cbc2b06a7d0dde166a90ae718d"
server_key_version: "encryptionkey@0"
I20260806 04:57:46.840449 15739 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785992227050689-9601-0/minicluster-data/master-0/wal/instance:
uuid: "3f6af880361b42bfb3fdfe2fd232c604"
format_stamp: "Formatted at 2026-08-06 04:57:46 on dist-test-slave-x4qh"
server_key: "6bef4d5844b65bdb1daba3dd664b5007"
server_key_iv: "63ffa0cbc2b06a7d0dde166a90ae718d"
server_key_version: "encryptionkey@0"
I20260806 04:57:46.845249 15739 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.003s
I20260806 04:57:46.848580 15770 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 04:57:46.849813 15739 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.004s
I20260806 04:57:46.849989 15739 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785992227050689-9601-0/minicluster-data/master-0/data,/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785992227050689-9601-0/minicluster-data/master-0/wal
uuid: "3f6af880361b42bfb3fdfe2fd232c604"
format_stamp: "Formatted at 2026-08-06 04:57:46 on dist-test-slave-x4qh"
server_key: "6bef4d5844b65bdb1daba3dd664b5007"
server_key_iv: "63ffa0cbc2b06a7d0dde166a90ae718d"
server_key_version: "encryptionkey@0"
I20260806 04:57:46.850215 15739 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785992227050689-9601-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785992227050689-9601-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785992227050689-9601-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 04:57:46.867996 15739 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 04:57:46.872228 15739 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 04:57:46.872517 15739 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 04:57:46.890738 15739 rpc_server.cc:307] RPC server started. Bound to: 127.9.96.126:36955
I20260806 04:57:46.890760 15854 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.96.126:36955 every 8 connection(s)
I20260806 04:57:46.891678 15739 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785992227050689-9601-0/minicluster-data/master-0/data/info.pb
I20260806 04:57:46.895262 15855 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 04:57:46.897882 9601 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu as pid 15739
I20260806 04:57:46.898113 9601 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785992227050689-9601-0/minicluster-data/master-0/wal/instance
I20260806 04:57:46.898766 9601 external_mini_cluster.cc:1428] Setting key 41c567726e9c71f1378189f74c617a2d
I20260806 04:57:46.907469 15855 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3f6af880361b42bfb3fdfe2fd232c604: Bootstrap starting.
I20260806 04:57:46.911293 15855 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3f6af880361b42bfb3fdfe2fd232c604: Neither blocks nor log segments found. Creating new log.
I20260806 04:57:46.912657 15855 log.cc:824] T 00000000000000000000000000000000 P 3f6af880361b42bfb3fdfe2fd232c604: Log is configured to *not* fsync() on all Append() calls
Aug 06 04:57:46 dist-test-slave-x4qh krb5kdc[15718](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992266, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.96.126@KRBTEST.COM
I20260806 04:57:46.915593 15855 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3f6af880361b42bfb3fdfe2fd232c604: No bootstrap required, opened a new log
I20260806 04:57:46.921909 15861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:57:46.900881 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59768 (local address 127.9.96.126:36955)
0806 04:57:46.901668 (+ 787us) server_negotiation.cc:206] Beginning negotiation
0806 04:57:46.901679 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 04:57:46.901718 (+ 39us) server_negotiation.cc:407] Connection header received
0806 04:57:46.903310 (+ 1592us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:57:46.903332 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:57:46.903778 (+ 446us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:57:46.904114 (+ 336us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:57:46.905989 (+ 1875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:57:46.907259 (+ 1270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:57:46.909107 (+ 1848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:57:46.909767 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:57:46.909916 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:57:46.914616 (+ 4700us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:57:46.914624 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:57:46.914639 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:57:46.914712 (+ 73us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:57:46.917821 (+ 3109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:57:46.918571 (+ 750us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:57:46.918585 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:57:46.918591 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:57:46.918773 (+ 182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:57:46.919164 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:57:46.919170 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:57:46.919173 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:57:46.919766 (+ 593us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:57:46.920098 (+ 332us) server_negotiation.cc:1035] Waiting for connection context
0806 04:57:46.920577 (+ 479us) server_negotiation.cc:299] Negotiation successful
0806 04:57:46.920977 (+ 400us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":569,"thread_start_us":367,"threads_started":1}
I20260806 04:57:46.923434 15855 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3f6af880361b42bfb3fdfe2fd232c604 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3f6af880361b42bfb3fdfe2fd232c604" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 36955 } }
I20260806 04:57:46.923799 15855 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3f6af880361b42bfb3fdfe2fd232c604 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 04:57:46.923934 15855 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3f6af880361b42bfb3fdfe2fd232c604 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f6af880361b42bfb3fdfe2fd232c604, State: Initialized, Role: FOLLOWER
I20260806 04:57:46.924520 15855 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3f6af880361b42bfb3fdfe2fd232c604 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3f6af880361b42bfb3fdfe2fd232c604" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 36955 } }
I20260806 04:57:46.924767 15855 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3f6af880361b42bfb3fdfe2fd232c604 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 04:57:46.924918 15855 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3f6af880361b42bfb3fdfe2fd232c604 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 04:57:46.925109 15855 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3f6af880361b42bfb3fdfe2fd232c604 [term 0 FOLLOWER]: Advancing to term 1
I20260806 04:57:46.927261 15855 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3f6af880361b42bfb3fdfe2fd232c604 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3f6af880361b42bfb3fdfe2fd232c604" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 36955 } }
I20260806 04:57:46.927861 15855 leader_election.cc:302] T 00000000000000000000000000000000 P 3f6af880361b42bfb3fdfe2fd232c604 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3f6af880361b42bfb3fdfe2fd232c604; no voters:
I20260806 04:57:46.928401 15855 leader_election.cc:288] T 00000000000000000000000000000000 P 3f6af880361b42bfb3fdfe2fd232c604 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 04:57:46.928530 15863 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3f6af880361b42bfb3fdfe2fd232c604 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 04:57:46.929271 15863 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3f6af880361b42bfb3fdfe2fd232c604 [term 1 LEADER]: Becoming Leader. State: Replica: 3f6af880361b42bfb3fdfe2fd232c604, State: Running, Role: LEADER
I20260806 04:57:46.929836 15863 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3f6af880361b42bfb3fdfe2fd232c604 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3f6af880361b42bfb3fdfe2fd232c604" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 36955 } }
I20260806 04:57:46.930554 15855 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3f6af880361b42bfb3fdfe2fd232c604 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 04:57:46.932571 15865 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3f6af880361b42bfb3fdfe2fd232c604 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3f6af880361b42bfb3fdfe2fd232c604. Latest consensus state: current_term: 1 leader_uuid: "3f6af880361b42bfb3fdfe2fd232c604" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3f6af880361b42bfb3fdfe2fd232c604" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 36955 } } }
I20260806 04:57:46.932662 15864 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3f6af880361b42bfb3fdfe2fd232c604 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3f6af880361b42bfb3fdfe2fd232c604" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3f6af880361b42bfb3fdfe2fd232c604" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 36955 } } }
I20260806 04:57:46.932924 15865 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3f6af880361b42bfb3fdfe2fd232c604 [sys.catalog]: This master's current role is: LEADER
I20260806 04:57:46.933033 15864 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3f6af880361b42bfb3fdfe2fd232c604 [sys.catalog]: This master's current role is: LEADER
I20260806 04:57:46.934489 15874 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 04:57:46.940989 15874 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 04:57:46.950241 15874 catalog_manager.cc:1347] Generated new cluster ID: 59ab3137a9874efc99fcf2ea3cb2e055
I20260806 04:57:46.950361 15874 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 04:57:47.007893 15874 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 04:57:47.009305 15874 catalog_manager.cc:1504] Loading token signing keys...
I20260806 04:57:47.020160 15874 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3f6af880361b42bfb3fdfe2fd232c604: Generated new TSK 0
I20260806 04:57:47.021024 15874 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.9.96.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.96.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.96.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.65.keytab.
Entry for principal kudu/127.9.96.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.65.keytab.
I20260806 04:57:47.083040 9601 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
/tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785992227050689-9601-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785992227050689-9601-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785992227050689-9601-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785992227050689-9601-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.96.65:0
--local_ip_for_outbound_sockets=127.9.96.65
--webserver_interface=127.9.96.65
--webserver_port=0
--tserver_master_addrs=127.9.96.126:36955
--keytab_file=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.65.keytab
--principal=kudu/127.9.96.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.96.84:39773
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785992227050689-9601-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785992227050689-9601-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785992227050689-9601-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 04:57:47.407222 15896 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 04:57:47.407581 15896 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 04:57:47.407765 15896 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 04:57:47.414839 15896 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 04:57:47.414976 15896 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 04:57:47.415153 15896 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.96.65
I20260806 04:57:47.424633 15896 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.96.84:39773
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785992227050689-9601-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785992227050689-9601-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.65.keytab
--principal=kudu/127.9.96.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.96.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785992227050689-9601-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.96.65
--webserver_port=0
--tserver_master_addrs=127.9.96.126:36955
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.96.65
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785992227050689-9601-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 03:56:12 UTC on ccycloud.quasar-guijbm.root.comops.site
build id 4569618
ASAN enabled
I20260806 04:57:47.425897 15896 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 04:57:47.427170 15896 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 04:57:47.441598 15906 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:57:47.441571 15902 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:57:47.442006 15904 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:57:47.442255 15896 server_base.cc:1034] running on GCE node
I20260806 04:57:47.442929 15896 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 04:57:47.443924 15896 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 04:57:47.445156 15896 hybrid_clock.cc:648] HybridClock initialized: now 1785992267445111 us; error 39 us; skew 500 ppm
I20260806 04:57:47.445482 15896 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 04:57:47 dist-test-slave-x4qh krb5kdc[15718](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992267, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.96.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 04:57:47.448624 15896 init.cc:377] Logged in from keytab as kudu/127.9.96.65@KRBTEST.COM (short username kudu)
I20260806 04:57:47.449663 15896 fs_manager.cc:359] Metadata directory not provided
I20260806 04:57:47.449752 15896 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 04:57:47.450008 15896 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 04:57:47.452838 15896 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785992227050689-9601-0/minicluster-data/ts-0/data/instance:
uuid: "422c9bcdb6044fb99fdfe5d55d83b545"
format_stamp: "Formatted at 2026-08-06 04:57:47 on dist-test-slave-x4qh"
server_key: "4e3526eaddfc1847a9932040dc8f7dbf"
server_key_iv: "23beccc86c4c64e84826da90bd34174f"
server_key_version: "encryptionkey@0"
I20260806 04:57:47.453666 15896 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785992227050689-9601-0/minicluster-data/ts-0/wal/instance:
uuid: "422c9bcdb6044fb99fdfe5d55d83b545"
format_stamp: "Formatted at 2026-08-06 04:57:47 on dist-test-slave-x4qh"
server_key: "4e3526eaddfc1847a9932040dc8f7dbf"
server_key_iv: "23beccc86c4c64e84826da90bd34174f"
server_key_version: "encryptionkey@0"
I20260806 04:57:47.458657 15896 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260806 04:57:47.461871 15915 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 04:57:47.463092 15896 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 04:57:47.463218 15896 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785992227050689-9601-0/minicluster-data/ts-0/data,/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785992227050689-9601-0/minicluster-data/ts-0/wal
uuid: "422c9bcdb6044fb99fdfe5d55d83b545"
format_stamp: "Formatted at 2026-08-06 04:57:47 on dist-test-slave-x4qh"
server_key: "4e3526eaddfc1847a9932040dc8f7dbf"
server_key_iv: "23beccc86c4c64e84826da90bd34174f"
server_key_version: "encryptionkey@0"
I20260806 04:57:47.463434 15896 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785992227050689-9601-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785992227050689-9601-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785992227050689-9601-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 04:57:47.480914 15896 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 04:57:47.484679 15896 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 04:57:47.484915 15896 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 04:57:47.485795 15896 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 04:57:47.487067 15896 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 04:57:47.487154 15896 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 04:57:47.487242 15896 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 04:57:47.487306 15896 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 04:57:47.525318 15896 rpc_server.cc:307] RPC server started. Bound to: 127.9.96.65:39883
I20260806 04:57:47.525367 16103 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.96.65:39883 every 8 connection(s)
I20260806 04:57:47.526188 15896 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785992227050689-9601-0/minicluster-data/ts-0/data/info.pb
I20260806 04:57:47.535528 9601 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu as pid 15896
I20260806 04:57:47.535737 9601 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785992227050689-9601-0/minicluster-data/ts-0/wal/instance
I20260806 04:57:47.536551 9601 external_mini_cluster.cc:1428] Setting key 641f0cc0f7d6326d83b90a6af6a55795
Aug 06 04:57:47 dist-test-slave-x4qh krb5kdc[15718](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992267, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.96.65@KRBTEST.COM for kudu/127.9.96.126@KRBTEST.COM
I20260806 04:57:47.547490 16106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:57:47.528514 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.96.65:40083 (local address 127.9.96.126:36955)
0806 04:57:47.528969 (+ 455us) server_negotiation.cc:206] Beginning negotiation
0806 04:57:47.528973 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 04:57:47.529744 (+ 771us) server_negotiation.cc:407] Connection header received
0806 04:57:47.531696 (+ 1952us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:57:47.531701 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:57:47.531791 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:57:47.531928 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:57:47.533343 (+ 1415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:57:47.534041 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:57:47.535588 (+ 1547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:57:47.536253 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:57:47.536394 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:57:47.540243 (+ 3849us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:57:47.540253 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:57:47.540261 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:57:47.540317 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:57:47.543912 (+ 3595us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:57:47.544810 (+ 898us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:57:47.544835 (+ 25us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:57:47.544841 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:57:47.545080 (+ 239us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:57:47.545717 (+ 637us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:57:47.545726 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:57:47.545732 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:57:47.546118 (+ 386us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:57:47.546274 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0806 04:57:47.546977 (+ 703us) server_negotiation.cc:299] Negotiation successful
0806 04:57:47.547148 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":345,"thread_start_us":247,"threads_started":1}
I20260806 04:57:47.548806 16107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:57:47.528905 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:36955 (local address 127.9.96.65:40083)
0806 04:57:47.529583 (+ 678us) negotiation.cc:107] Waiting for socket to connect
0806 04:57:47.529618 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0806 04:57:47.531433 (+ 1815us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:57:47.532122 (+ 689us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:57:47.532130 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:57:47.532643 (+ 513us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:57:47.533191 (+ 548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:57:47.533208 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:57:47.534169 (+ 961us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:57:47.534172 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:57:47.535428 (+ 1256us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:57:47.535444 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:57:47.536450 (+ 1006us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:57:47.536455 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:57:47.536674 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:57:47.537480 (+ 806us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:57:47.537511 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:57:47.540012 (+ 2501us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 04:57:47.544040 (+ 4028us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:57:47.544047 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:57:47.544057 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:57:47.544579 (+ 522us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:57:47.545325 (+ 746us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:57:47.545329 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:57:47.545332 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:57:47.545555 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:57:47.546256 (+ 701us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 04:57:47.546262 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 04:57:47.546554 (+ 292us) client_negotiation.cc:769] Sending connection context
0806 04:57:47.546804 (+ 250us) client_negotiation.cc:240] Negotiation successful
0806 04:57:47.547365 (+ 561us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":458,"thread_start_us":266,"threads_started":1}
I20260806 04:57:47.550397 16104 heartbeater.cc:347] Connected to a master server at 127.9.96.126:36955
I20260806 04:57:47.550693 16104 heartbeater.cc:464] Registering TS with master...
I20260806 04:57:47.551306 16104 heartbeater.cc:511] Master 127.9.96.126:36955 requested a full tablet report, sending...
I20260806 04:57:47.553431 15799 ts_manager.cc:194] Registered new tserver with Master: 422c9bcdb6044fb99fdfe5d55d83b545 (127.9.96.65:39883)
I20260806 04:57:47.555706 15799 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.96.65@KRBTEST.COM'} at 127.9.96.65:40083
WARNING: no policy specified for kudu/127.9.96.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.96.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.96.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.66.keytab.
Entry for principal kudu/127.9.96.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.66.keytab.
I20260806 04:57:47.588621 9601 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
/tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785992227050689-9601-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785992227050689-9601-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785992227050689-9601-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785992227050689-9601-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.96.66:0
--local_ip_for_outbound_sockets=127.9.96.66
--webserver_interface=127.9.96.66
--webserver_port=0
--tserver_master_addrs=127.9.96.126:36955
--keytab_file=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.66.keytab
--principal=kudu/127.9.96.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.96.84:39773
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785992227050689-9601-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785992227050689-9601-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785992227050689-9601-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 04:57:47.919284 16111 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 04:57:47.919629 16111 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 04:57:47.919832 16111 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 04:57:47.927106 16111 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 04:57:47.927253 16111 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 04:57:47.927488 16111 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.96.66
I20260806 04:57:47.937363 16111 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.96.84:39773
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785992227050689-9601-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785992227050689-9601-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.66.keytab
--principal=kudu/127.9.96.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.96.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785992227050689-9601-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.96.66
--webserver_port=0
--tserver_master_addrs=127.9.96.126:36955
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.96.66
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785992227050689-9601-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 03:56:12 UTC on ccycloud.quasar-guijbm.root.comops.site
build id 4569618
ASAN enabled
I20260806 04:57:47.938764 16111 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 04:57:47.940145 16111 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 04:57:47.953457 16120 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:57:47.953374 16121 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:57:47.953426 16128 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:57:47.954552 16111 server_base.cc:1034] running on GCE node
I20260806 04:57:47.955165 16111 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 04:57:47.956151 16111 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 04:57:47.957417 16111 hybrid_clock.cc:648] HybridClock initialized: now 1785992267957369 us; error 54 us; skew 500 ppm
I20260806 04:57:47.957753 16111 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 04:57:47 dist-test-slave-x4qh krb5kdc[15718](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992267, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.96.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 04:57:47.960860 16111 init.cc:377] Logged in from keytab as kudu/127.9.96.66@KRBTEST.COM (short username kudu)
I20260806 04:57:47.962083 16111 fs_manager.cc:359] Metadata directory not provided
I20260806 04:57:47.962220 16111 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 04:57:47.962553 16111 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 04:57:47.965535 16111 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785992227050689-9601-0/minicluster-data/ts-1/data/instance:
uuid: "a1ce135e8863478183660033cf89cab3"
format_stamp: "Formatted at 2026-08-06 04:57:47 on dist-test-slave-x4qh"
server_key: "88569e18b3ed749ec8432e858a5b16fe"
server_key_iv: "2011be27448f6e78eaeb8d4d9b56212a"
server_key_version: "encryptionkey@0"
I20260806 04:57:47.966434 16111 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785992227050689-9601-0/minicluster-data/ts-1/wal/instance:
uuid: "a1ce135e8863478183660033cf89cab3"
format_stamp: "Formatted at 2026-08-06 04:57:47 on dist-test-slave-x4qh"
server_key: "88569e18b3ed749ec8432e858a5b16fe"
server_key_iv: "2011be27448f6e78eaeb8d4d9b56212a"
server_key_version: "encryptionkey@0"
I20260806 04:57:47.971606 16111 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260806 04:57:47.974999 16139 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 04:57:47.976315 16111 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 04:57:47.976516 16111 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785992227050689-9601-0/minicluster-data/ts-1/data,/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785992227050689-9601-0/minicluster-data/ts-1/wal
uuid: "a1ce135e8863478183660033cf89cab3"
format_stamp: "Formatted at 2026-08-06 04:57:47 on dist-test-slave-x4qh"
server_key: "88569e18b3ed749ec8432e858a5b16fe"
server_key_iv: "2011be27448f6e78eaeb8d4d9b56212a"
server_key_version: "encryptionkey@0"
I20260806 04:57:47.976765 16111 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785992227050689-9601-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785992227050689-9601-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785992227050689-9601-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 04:57:48.011926 16111 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 04:57:48.016062 16111 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 04:57:48.016355 16111 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 04:57:48.017391 16111 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 04:57:48.018752 16111 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 04:57:48.018878 16111 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 04:57:48.019022 16111 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 04:57:48.019119 16111 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 04:57:48.053112 16111 rpc_server.cc:307] RPC server started. Bound to: 127.9.96.66:40759
I20260806 04:57:48.053169 16313 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.96.66:40759 every 8 connection(s)
I20260806 04:57:48.054019 16111 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785992227050689-9601-0/minicluster-data/ts-1/data/info.pb
I20260806 04:57:48.059562 9601 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu as pid 16111
I20260806 04:57:48.059743 9601 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785992227050689-9601-0/minicluster-data/ts-1/wal/instance
I20260806 04:57:48.060212 9601 external_mini_cluster.cc:1428] Setting key a27cb43299c75eb4e26904afa0713cd4
Aug 06 04:57:48 dist-test-slave-x4qh krb5kdc[15718](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992267, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.96.66@KRBTEST.COM for kudu/127.9.96.126@KRBTEST.COM
I20260806 04:57:48.074712 16317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:57:48.056581 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.96.66:55445 (local address 127.9.96.126:36955)
0806 04:57:48.057017 (+ 436us) server_negotiation.cc:206] Beginning negotiation
0806 04:57:48.057021 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 04:57:48.058007 (+ 986us) server_negotiation.cc:407] Connection header received
0806 04:57:48.059864 (+ 1857us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:57:48.059870 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:57:48.059961 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:57:48.060099 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:57:48.062028 (+ 1929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:57:48.062756 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:57:48.064392 (+ 1636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:57:48.064847 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:57:48.064983 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:57:48.068652 (+ 3669us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:57:48.068657 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:57:48.068661 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:57:48.068693 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:57:48.071357 (+ 2664us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:57:48.072375 (+ 1018us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:57:48.072378 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:57:48.072381 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:57:48.072511 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:57:48.073001 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:57:48.073005 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:57:48.073007 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:57:48.073274 (+ 267us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:57:48.073394 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0806 04:57:48.074290 (+ 896us) server_negotiation.cc:299] Negotiation successful
0806 04:57:48.074461 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":329,"thread_start_us":218,"threads_started":1}
I20260806 04:57:48.076329 16318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:57:48.057127 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:36955 (local address 127.9.96.66:55445)
0806 04:57:48.057836 (+ 709us) negotiation.cc:107] Waiting for socket to connect
0806 04:57:48.057888 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0806 04:57:48.059575 (+ 1687us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:57:48.060364 (+ 789us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:57:48.060375 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:57:48.061051 (+ 676us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:57:48.061812 (+ 761us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:57:48.061836 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:57:48.062925 (+ 1089us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:57:48.062930 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:57:48.064192 (+ 1262us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:57:48.064204 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:57:48.065084 (+ 880us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:57:48.065091 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:57:48.065322 (+ 231us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:57:48.066042 (+ 720us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:57:48.066065 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:57:48.068424 (+ 2359us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 04:57:48.071557 (+ 3133us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:57:48.071570 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:57:48.071590 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:57:48.072245 (+ 655us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:57:48.072623 (+ 378us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:57:48.072630 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:57:48.072636 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:57:48.072888 (+ 252us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:57:48.073422 (+ 534us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 04:57:48.073432 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 04:57:48.073903 (+ 471us) client_negotiation.cc:769] Sending connection context
0806 04:57:48.074213 (+ 310us) client_negotiation.cc:240] Negotiation successful
0806 04:57:48.074776 (+ 563us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":491,"thread_start_us":306,"threads_started":1}
I20260806 04:57:48.077888 16314 heartbeater.cc:347] Connected to a master server at 127.9.96.126:36955
I20260806 04:57:48.078203 16314 heartbeater.cc:464] Registering TS with master...
I20260806 04:57:48.078855 16314 heartbeater.cc:511] Master 127.9.96.126:36955 requested a full tablet report, sending...
I20260806 04:57:48.080418 15799 ts_manager.cc:194] Registered new tserver with Master: a1ce135e8863478183660033cf89cab3 (127.9.96.66:40759)
I20260806 04:57:48.081383 15799 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.96.66@KRBTEST.COM'} at 127.9.96.66:55445
WARNING: no policy specified for kudu/127.9.96.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.96.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.96.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.67.keytab.
Entry for principal kudu/127.9.96.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.67.keytab.
I20260806 04:57:48.112143 9601 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
/tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785992227050689-9601-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785992227050689-9601-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785992227050689-9601-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785992227050689-9601-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.96.67:0
--local_ip_for_outbound_sockets=127.9.96.67
--webserver_interface=127.9.96.67
--webserver_port=0
--tserver_master_addrs=127.9.96.126:36955
--keytab_file=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.67.keytab
--principal=kudu/127.9.96.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.96.84:39773
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785992227050689-9601-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785992227050689-9601-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785992227050689-9601-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 04:57:48.432490 16322 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 04:57:48.432729 16322 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 04:57:48.432828 16322 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 04:57:48.441325 16322 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 04:57:48.441422 16322 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 04:57:48.441532 16322 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.96.67
I20260806 04:57:48.451047 16322 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.96.84:39773
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785992227050689-9601-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785992227050689-9601-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.67.keytab
--principal=kudu/127.9.96.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.96.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785992227050689-9601-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.96.67
--webserver_port=0
--tserver_master_addrs=127.9.96.126:36955
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.96.67
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785992227050689-9601-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 03:56:12 UTC on ccycloud.quasar-guijbm.root.comops.site
build id 4569618
ASAN enabled
I20260806 04:57:48.452301 16322 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 04:57:48.453650 16322 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 04:57:48.466813 16330 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:57:48.466979 16332 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:57:48.467793 16328 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:57:48.467824 16322 server_base.cc:1034] running on GCE node
I20260806 04:57:48.468524 16322 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 04:57:48.469384 16322 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 04:57:48.470582 16322 hybrid_clock.cc:648] HybridClock initialized: now 1785992268470540 us; error 30 us; skew 500 ppm
I20260806 04:57:48.470920 16322 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 04:57:48 dist-test-slave-x4qh krb5kdc[15718](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992268, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.96.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 04:57:48.474045 16322 init.cc:377] Logged in from keytab as kudu/127.9.96.67@KRBTEST.COM (short username kudu)
I20260806 04:57:48.475275 16322 fs_manager.cc:359] Metadata directory not provided
I20260806 04:57:48.475446 16322 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 04:57:48.475782 16322 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 04:57:48.478670 16322 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785992227050689-9601-0/minicluster-data/ts-2/data/instance:
uuid: "c564852e7c874383938f7800a189926a"
format_stamp: "Formatted at 2026-08-06 04:57:48 on dist-test-slave-x4qh"
server_key: "645f291b05243fbd643a88fad2fa43ad"
server_key_iv: "abdffb4e54d941c991280180004fb685"
server_key_version: "encryptionkey@0"
I20260806 04:57:48.479580 16322 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785992227050689-9601-0/minicluster-data/ts-2/wal/instance:
uuid: "c564852e7c874383938f7800a189926a"
format_stamp: "Formatted at 2026-08-06 04:57:48 on dist-test-slave-x4qh"
server_key: "645f291b05243fbd643a88fad2fa43ad"
server_key_iv: "abdffb4e54d941c991280180004fb685"
server_key_version: "encryptionkey@0"
I20260806 04:57:48.484907 16322 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260806 04:57:48.488399 16340 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 04:57:48.489653 16322 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 04:57:48.489840 16322 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785992227050689-9601-0/minicluster-data/ts-2/data,/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785992227050689-9601-0/minicluster-data/ts-2/wal
uuid: "c564852e7c874383938f7800a189926a"
format_stamp: "Formatted at 2026-08-06 04:57:48 on dist-test-slave-x4qh"
server_key: "645f291b05243fbd643a88fad2fa43ad"
server_key_iv: "abdffb4e54d941c991280180004fb685"
server_key_version: "encryptionkey@0"
I20260806 04:57:48.490062 16322 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785992227050689-9601-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785992227050689-9601-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785992227050689-9601-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 04:57:48.500782 16322 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 04:57:48.504423 16322 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 04:57:48.504700 16322 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 04:57:48.505568 16322 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 04:57:48.506894 16322 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 04:57:48.507015 16322 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 04:57:48.507144 16322 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 04:57:48.507242 16322 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 04:57:48.543450 16322 rpc_server.cc:307] RPC server started. Bound to: 127.9.96.67:46759
I20260806 04:57:48.543529 16508 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.96.67:46759 every 8 connection(s)
I20260806 04:57:48.544271 16322 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785992227050689-9601-0/minicluster-data/ts-2/data/info.pb
I20260806 04:57:48.551715 9601 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu as pid 16322
I20260806 04:57:48.551885 9601 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785992227050689-9601-0/minicluster-data/ts-2/wal/instance
I20260806 04:57:48.552325 9601 external_mini_cluster.cc:1428] Setting key 4e7503312f0e15974e10a2d0f8d06987
Aug 06 04:57:48 dist-test-slave-x4qh krb5kdc[15718](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992268, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.96.67@KRBTEST.COM for kudu/127.9.96.126@KRBTEST.COM
I20260806 04:57:48.558485 16104 heartbeater.cc:503] Master 127.9.96.126:36955 was elected leader, sending a full tablet report...
I20260806 04:57:48.564703 16317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:57:48.546637 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.96.67:46845 (local address 127.9.96.126:36955)
0806 04:57:48.546847 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0806 04:57:48.546853 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 04:57:48.547998 (+ 1145us) server_negotiation.cc:407] Connection header received
0806 04:57:48.549807 (+ 1809us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:57:48.549813 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:57:48.549900 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:57:48.550022 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:57:48.551595 (+ 1573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:57:48.552497 (+ 902us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:57:48.554603 (+ 2106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:57:48.555024 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:57:48.555155 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:57:48.558505 (+ 3350us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:57:48.558512 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:57:48.558518 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:57:48.558556 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:57:48.561229 (+ 2673us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:57:48.562268 (+ 1039us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:57:48.562280 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:57:48.562282 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:57:48.562413 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:57:48.562855 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:57:48.562859 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:57:48.562862 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:57:48.563133 (+ 271us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:57:48.563307 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0806 04:57:48.564147 (+ 840us) server_negotiation.cc:299] Negotiation successful
0806 04:57:48.564375 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260806 04:57:48.565559 16511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:57:48.547099 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:36955 (local address 127.9.96.67:46845)
0806 04:57:48.547844 (+ 745us) negotiation.cc:107] Waiting for socket to connect
0806 04:57:48.547889 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0806 04:57:48.549532 (+ 1643us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:57:48.550260 (+ 728us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:57:48.550268 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:57:48.550745 (+ 477us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:57:48.551347 (+ 602us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:57:48.551376 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:57:48.552651 (+ 1275us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:57:48.552658 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:57:48.554439 (+ 1781us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:57:48.554453 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:57:48.555236 (+ 783us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:57:48.555243 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:57:48.555522 (+ 279us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:57:48.556183 (+ 661us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:57:48.556204 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:57:48.558297 (+ 2093us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 04:57:48.561382 (+ 3085us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:57:48.561390 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:57:48.561403 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:57:48.562128 (+ 725us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:57:48.562531 (+ 403us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:57:48.562536 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:57:48.562540 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:57:48.562723 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:57:48.563284 (+ 561us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 04:57:48.563291 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 04:57:48.563731 (+ 440us) client_negotiation.cc:769] Sending connection context
0806 04:57:48.564047 (+ 316us) client_negotiation.cc:240] Negotiation successful
0806 04:57:48.564533 (+ 486us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":526,"thread_start_us":349,"threads_started":1}
I20260806 04:57:48.567035 16509 heartbeater.cc:347] Connected to a master server at 127.9.96.126:36955
I20260806 04:57:48.567332 16509 heartbeater.cc:464] Registering TS with master...
I20260806 04:57:48.568063 16509 heartbeater.cc:511] Master 127.9.96.126:36955 requested a full tablet report, sending...
I20260806 04:57:48.569427 15798 ts_manager.cc:194] Registered new tserver with Master: c564852e7c874383938f7800a189926a (127.9.96.67:46759)
I20260806 04:57:48.570225 15798 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.96.67@KRBTEST.COM'} at 127.9.96.67:46845
I20260806 04:57:48.579735 9601 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 04:57:48.596064 16317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:57:48.584270 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59774 (local address 127.9.96.126:36955)
0806 04:57:48.584420 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0806 04:57:48.584424 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 04:57:48.584953 (+ 529us) server_negotiation.cc:407] Connection header received
0806 04:57:48.585078 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:57:48.585082 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:57:48.585148 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:57:48.585225 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:57:48.586592 (+ 1367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:57:48.587269 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:57:48.588481 (+ 1212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:57:48.588897 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:57:48.588989 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:57:48.591166 (+ 2177us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:57:48.591170 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:57:48.591174 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:57:48.591198 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:57:48.593562 (+ 2364us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:57:48.594222 (+ 660us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:57:48.594233 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:57:48.594248 (+ 15us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:57:48.594370 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:57:48.594699 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:57:48.594702 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:57:48.594704 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:57:48.595025 (+ 321us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:57:48.595231 (+ 206us) server_negotiation.cc:1035] Waiting for connection context
0806 04:57:48.595540 (+ 309us) server_negotiation.cc:299] Negotiation successful
0806 04:57:48.595702 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260806 04:57:48.599967 15798 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59774:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 04:57:48.606603 15798 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 04:57:48.629181 16525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:57:48.618373 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.65:39883 (local address 127.0.0.1:54948)
0806 04:57:48.619642 (+ 1269us) negotiation.cc:107] Waiting for socket to connect
0806 04:57:48.619676 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0806 04:57:48.619826 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:57:48.620822 (+ 996us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:57:48.620829 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:57:48.620861 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 04:57:48.621940 (+ 1079us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:57:48.621952 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:57:48.623987 (+ 2035us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:57:48.623991 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:57:48.626420 (+ 2429us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:57:48.626437 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:57:48.628145 (+ 1708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:57:48.628151 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:57:48.628376 (+ 225us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:57:48.628419 (+ 43us) client_negotiation.cc:769] Sending connection context
0806 04:57:48.628588 (+ 169us) client_negotiation.cc:240] Negotiation successful
0806 04:57:48.628772 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1082,"mutex_wait_us":1,"thread_start_us":928,"threads_started":1}
I20260806 04:57:48.629743 16528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:57:48.620054 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.66:40759 (local address 127.0.0.1:45360)
0806 04:57:48.620564 (+ 510us) negotiation.cc:107] Waiting for socket to connect
0806 04:57:48.620585 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0806 04:57:48.620728 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:57:48.622363 (+ 1635us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:57:48.622368 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:57:48.622401 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 04:57:48.622936 (+ 535us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:57:48.622946 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:57:48.625036 (+ 2090us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:57:48.625042 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:57:48.626976 (+ 1934us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:57:48.626991 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:57:48.628857 (+ 1866us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:57:48.628864 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:57:48.629036 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:57:48.629070 (+ 34us) client_negotiation.cc:769] Sending connection context
0806 04:57:48.629154 (+ 84us) client_negotiation.cc:240] Negotiation successful
0806 04:57:48.629283 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":387,"mutex_wait_us":1,"spinlock_wait_cycles":17664,"thread_start_us":261,"threads_started":1}
I20260806 04:57:48.630105 16526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:57:48.618524 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54948 (local address 127.9.96.65:39883)
0806 04:57:48.619398 (+ 874us) server_negotiation.cc:206] Beginning negotiation
0806 04:57:48.619407 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 04:57:48.619789 (+ 382us) server_negotiation.cc:407] Connection header received
0806 04:57:48.620023 (+ 234us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:57:48.620032 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:57:48.620303 (+ 271us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:57:48.620517 (+ 214us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 04:57:48.622148 (+ 1631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:57:48.623823 (+ 1675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:57:48.626601 (+ 2778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:57:48.627944 (+ 1343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:57:48.628084 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:57:48.628325 (+ 241us) server_negotiation.cc:1035] Waiting for connection context
0806 04:57:48.628625 (+ 300us) server_negotiation.cc:299] Negotiation successful
0806 04:57:48.628782 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":636,"thread_start_us":435,"threads_started":1}
I20260806 04:57:48.631073 16529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:57:48.620397 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45360 (local address 127.9.96.66:40759)
0806 04:57:48.621672 (+ 1275us) server_negotiation.cc:206] Beginning negotiation
0806 04:57:48.621684 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 04:57:48.621719 (+ 35us) server_negotiation.cc:407] Connection header received
0806 04:57:48.621830 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:57:48.621839 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:57:48.622109 (+ 270us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:57:48.623114 (+ 1005us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 04:57:48.623301 (+ 187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:57:48.624832 (+ 1531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:57:48.627151 (+ 2319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:57:48.628619 (+ 1468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:57:48.630329 (+ 1710us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:57:48.630532 (+ 203us) server_negotiation.cc:1035] Waiting for connection context
0806 04:57:48.630596 (+ 64us) server_negotiation.cc:299] Negotiation successful
0806 04:57:48.630769 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1067,"thread_start_us":916,"threads_started":1}
I20260806 04:57:48.633205 16530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:57:48.620640 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.67:46759 (local address 127.0.0.1:36844)
0806 04:57:48.622481 (+ 1841us) negotiation.cc:107] Waiting for socket to connect
0806 04:57:48.622500 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0806 04:57:48.622631 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:57:48.623736 (+ 1105us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:57:48.623741 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:57:48.623766 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 04:57:48.624159 (+ 393us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:57:48.624173 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:57:48.626635 (+ 2462us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:57:48.626643 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:57:48.628696 (+ 2053us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:57:48.628711 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:57:48.632402 (+ 3691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:57:48.632409 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:57:48.632583 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:57:48.632614 (+ 31us) client_negotiation.cc:769] Sending connection context
0806 04:57:48.632718 (+ 104us) client_negotiation.cc:240] Negotiation successful
0806 04:57:48.632844 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":572,"spinlock_wait_cycles":896,"thread_start_us":396,"threads_started":1}
I20260806 04:57:48.633255 16531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:57:48.621221 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36844 (local address 127.9.96.67:46759)
0806 04:57:48.623156 (+ 1935us) server_negotiation.cc:206] Beginning negotiation
0806 04:57:48.623166 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 04:57:48.623195 (+ 29us) server_negotiation.cc:407] Connection header received
0806 04:57:48.623295 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:57:48.623313 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:57:48.623555 (+ 242us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:57:48.623719 (+ 164us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 04:57:48.624406 (+ 687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:57:48.626524 (+ 2118us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:57:48.628889 (+ 2365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:57:48.632190 (+ 3301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:57:48.632338 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:57:48.632546 (+ 208us) server_negotiation.cc:1035] Waiting for connection context
0806 04:57:48.632770 (+ 224us) server_negotiation.cc:299] Negotiation successful
0806 04:57:48.632927 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1785,"thread_start_us":1642,"threads_started":1}
I20260806 04:57:48.634116 15994 tablet_service.cc:1467] Processing CreateTablet for tablet 74a243b846bf4c48a1c4d1bcbb356b01 (DEFAULT_TABLE table=test-table [id=29e0f2ae980a43d8b81d4ccebb400bb3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 04:57:48.634495 16213 tablet_service.cc:1467] Processing CreateTablet for tablet 74a243b846bf4c48a1c4d1bcbb356b01 (DEFAULT_TABLE table=test-table [id=29e0f2ae980a43d8b81d4ccebb400bb3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 04:57:48.635552 15994 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 74a243b846bf4c48a1c4d1bcbb356b01. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 04:57:48.635898 16213 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 74a243b846bf4c48a1c4d1bcbb356b01. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 04:57:48.636941 16416 tablet_service.cc:1467] Processing CreateTablet for tablet 74a243b846bf4c48a1c4d1bcbb356b01 (DEFAULT_TABLE table=test-table [id=29e0f2ae980a43d8b81d4ccebb400bb3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 04:57:48.638262 16416 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 74a243b846bf4c48a1c4d1bcbb356b01. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 04:57:48.647076 16532 tablet_bootstrap.cc:492] T 74a243b846bf4c48a1c4d1bcbb356b01 P 422c9bcdb6044fb99fdfe5d55d83b545: Bootstrap starting.
I20260806 04:57:48.650183 16533 tablet_bootstrap.cc:492] T 74a243b846bf4c48a1c4d1bcbb356b01 P a1ce135e8863478183660033cf89cab3: Bootstrap starting.
I20260806 04:57:48.651013 16532 tablet_bootstrap.cc:654] T 74a243b846bf4c48a1c4d1bcbb356b01 P 422c9bcdb6044fb99fdfe5d55d83b545: Neither blocks nor log segments found. Creating new log.
I20260806 04:57:48.652143 16534 tablet_bootstrap.cc:492] T 74a243b846bf4c48a1c4d1bcbb356b01 P c564852e7c874383938f7800a189926a: Bootstrap starting.
I20260806 04:57:48.652334 16532 log.cc:824] T 74a243b846bf4c48a1c4d1bcbb356b01 P 422c9bcdb6044fb99fdfe5d55d83b545: Log is configured to *not* fsync() on all Append() calls
I20260806 04:57:48.653658 16533 tablet_bootstrap.cc:654] T 74a243b846bf4c48a1c4d1bcbb356b01 P a1ce135e8863478183660033cf89cab3: Neither blocks nor log segments found. Creating new log.
I20260806 04:57:48.655026 16533 log.cc:824] T 74a243b846bf4c48a1c4d1bcbb356b01 P a1ce135e8863478183660033cf89cab3: Log is configured to *not* fsync() on all Append() calls
I20260806 04:57:48.655588 16534 tablet_bootstrap.cc:654] T 74a243b846bf4c48a1c4d1bcbb356b01 P c564852e7c874383938f7800a189926a: Neither blocks nor log segments found. Creating new log.
I20260806 04:57:48.655738 16532 tablet_bootstrap.cc:492] T 74a243b846bf4c48a1c4d1bcbb356b01 P 422c9bcdb6044fb99fdfe5d55d83b545: No bootstrap required, opened a new log
I20260806 04:57:48.655989 16532 ts_tablet_manager.cc:1397] T 74a243b846bf4c48a1c4d1bcbb356b01 P 422c9bcdb6044fb99fdfe5d55d83b545: Time spent bootstrapping tablet: real 0.009s user 0.002s sys 0.006s
I20260806 04:57:48.656929 16534 log.cc:824] T 74a243b846bf4c48a1c4d1bcbb356b01 P c564852e7c874383938f7800a189926a: Log is configured to *not* fsync() on all Append() calls
I20260806 04:57:48.658349 16533 tablet_bootstrap.cc:492] T 74a243b846bf4c48a1c4d1bcbb356b01 P a1ce135e8863478183660033cf89cab3: No bootstrap required, opened a new log
I20260806 04:57:48.658599 16533 ts_tablet_manager.cc:1397] T 74a243b846bf4c48a1c4d1bcbb356b01 P a1ce135e8863478183660033cf89cab3: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260806 04:57:48.665349 16534 tablet_bootstrap.cc:492] T 74a243b846bf4c48a1c4d1bcbb356b01 P c564852e7c874383938f7800a189926a: No bootstrap required, opened a new log
I20260806 04:57:48.665607 16534 ts_tablet_manager.cc:1397] T 74a243b846bf4c48a1c4d1bcbb356b01 P c564852e7c874383938f7800a189926a: Time spent bootstrapping tablet: real 0.014s user 0.003s sys 0.009s
I20260806 04:57:48.666496 16532 raft_consensus.cc:348] T 74a243b846bf4c48a1c4d1bcbb356b01 P 422c9bcdb6044fb99fdfe5d55d83b545 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "422c9bcdb6044fb99fdfe5d55d83b545" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 39883 } } peers { permanent_uuid: "a1ce135e8863478183660033cf89cab3" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 40759 } } peers { permanent_uuid: "c564852e7c874383938f7800a189926a" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 46759 } }
I20260806 04:57:48.666971 16532 raft_consensus.cc:374] T 74a243b846bf4c48a1c4d1bcbb356b01 P 422c9bcdb6044fb99fdfe5d55d83b545 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 04:57:48.667141 16532 raft_consensus.cc:729] T 74a243b846bf4c48a1c4d1bcbb356b01 P 422c9bcdb6044fb99fdfe5d55d83b545 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 422c9bcdb6044fb99fdfe5d55d83b545, State: Initialized, Role: FOLLOWER
I20260806 04:57:48.667826 16532 consensus_queue.cc:260] T 74a243b846bf4c48a1c4d1bcbb356b01 P 422c9bcdb6044fb99fdfe5d55d83b545 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "422c9bcdb6044fb99fdfe5d55d83b545" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 39883 } } peers { permanent_uuid: "a1ce135e8863478183660033cf89cab3" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 40759 } } peers { permanent_uuid: "c564852e7c874383938f7800a189926a" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 46759 } }
I20260806 04:57:48.669170 16533 raft_consensus.cc:348] T 74a243b846bf4c48a1c4d1bcbb356b01 P a1ce135e8863478183660033cf89cab3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "422c9bcdb6044fb99fdfe5d55d83b545" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 39883 } } peers { permanent_uuid: "a1ce135e8863478183660033cf89cab3" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 40759 } } peers { permanent_uuid: "c564852e7c874383938f7800a189926a" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 46759 } }
I20260806 04:57:48.669564 16533 raft_consensus.cc:374] T 74a243b846bf4c48a1c4d1bcbb356b01 P a1ce135e8863478183660033cf89cab3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 04:57:48.669689 16532 ts_tablet_manager.cc:1428] T 74a243b846bf4c48a1c4d1bcbb356b01 P 422c9bcdb6044fb99fdfe5d55d83b545: Time spent starting tablet: real 0.014s user 0.012s sys 0.001s
I20260806 04:57:48.669690 16533 raft_consensus.cc:729] T 74a243b846bf4c48a1c4d1bcbb356b01 P a1ce135e8863478183660033cf89cab3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a1ce135e8863478183660033cf89cab3, State: Initialized, Role: FOLLOWER
I20260806 04:57:48.670245 16533 consensus_queue.cc:260] T 74a243b846bf4c48a1c4d1bcbb356b01 P a1ce135e8863478183660033cf89cab3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "422c9bcdb6044fb99fdfe5d55d83b545" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 39883 } } peers { permanent_uuid: "a1ce135e8863478183660033cf89cab3" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 40759 } } peers { permanent_uuid: "c564852e7c874383938f7800a189926a" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 46759 } }
I20260806 04:57:48.671406 16314 heartbeater.cc:503] Master 127.9.96.126:36955 was elected leader, sending a full tablet report...
I20260806 04:57:48.671890 16533 ts_tablet_manager.cc:1428] T 74a243b846bf4c48a1c4d1bcbb356b01 P a1ce135e8863478183660033cf89cab3: Time spent starting tablet: real 0.013s user 0.014s sys 0.000s
I20260806 04:57:48.676306 16534 raft_consensus.cc:348] T 74a243b846bf4c48a1c4d1bcbb356b01 P c564852e7c874383938f7800a189926a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "422c9bcdb6044fb99fdfe5d55d83b545" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 39883 } } peers { permanent_uuid: "a1ce135e8863478183660033cf89cab3" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 40759 } } peers { permanent_uuid: "c564852e7c874383938f7800a189926a" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 46759 } }
I20260806 04:57:48.676654 16534 raft_consensus.cc:374] T 74a243b846bf4c48a1c4d1bcbb356b01 P c564852e7c874383938f7800a189926a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 04:57:48.676764 16534 raft_consensus.cc:729] T 74a243b846bf4c48a1c4d1bcbb356b01 P c564852e7c874383938f7800a189926a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c564852e7c874383938f7800a189926a, State: Initialized, Role: FOLLOWER
I20260806 04:57:48.677312 16534 consensus_queue.cc:260] T 74a243b846bf4c48a1c4d1bcbb356b01 P c564852e7c874383938f7800a189926a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "422c9bcdb6044fb99fdfe5d55d83b545" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 39883 } } peers { permanent_uuid: "a1ce135e8863478183660033cf89cab3" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 40759 } } peers { permanent_uuid: "c564852e7c874383938f7800a189926a" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 46759 } }
I20260806 04:57:48.678264 16509 heartbeater.cc:503] Master 127.9.96.126:36955 was elected leader, sending a full tablet report...
I20260806 04:57:48.678783 16534 ts_tablet_manager.cc:1428] T 74a243b846bf4c48a1c4d1bcbb356b01 P c564852e7c874383938f7800a189926a: Time spent starting tablet: real 0.013s user 0.005s sys 0.007s
I20260806 04:57:48.685060 16538 raft_consensus.cc:482] T 74a243b846bf4c48a1c4d1bcbb356b01 P 422c9bcdb6044fb99fdfe5d55d83b545 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 04:57:48.685250 16538 raft_consensus.cc:504] T 74a243b846bf4c48a1c4d1bcbb356b01 P 422c9bcdb6044fb99fdfe5d55d83b545 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "422c9bcdb6044fb99fdfe5d55d83b545" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 39883 } } peers { permanent_uuid: "a1ce135e8863478183660033cf89cab3" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 40759 } } peers { permanent_uuid: "c564852e7c874383938f7800a189926a" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 46759 } }
I20260806 04:57:48.686622 16538 leader_election.cc:288] T 74a243b846bf4c48a1c4d1bcbb356b01 P 422c9bcdb6044fb99fdfe5d55d83b545 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a1ce135e8863478183660033cf89cab3 (127.9.96.66:40759), c564852e7c874383938f7800a189926a (127.9.96.67:46759)
I20260806 04:57:48.694489 16529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:57:48.686864 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.96.65:33819 (local address 127.9.96.66:40759)
0806 04:57:48.687025 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0806 04:57:48.687030 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 04:57:48.687673 (+ 643us) server_negotiation.cc:407] Connection header received
0806 04:57:48.687884 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:57:48.687889 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:57:48.687962 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:57:48.688065 (+ 103us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 04:57:48.688742 (+ 677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:57:48.689751 (+ 1009us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:57:48.692292 (+ 2541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:57:48.693553 (+ 1261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:57:48.693697 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:57:48.693817 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0806 04:57:48.694099 (+ 282us) server_negotiation.cc:299] Negotiation successful
0806 04:57:48.694225 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260806 04:57:48.694667 16544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:57:48.686934 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.66:40759 (local address 127.9.96.65:33819)
0806 04:57:48.687545 (+ 611us) negotiation.cc:107] Waiting for socket to connect
0806 04:57:48.687561 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 04:57:48.687728 (+ 167us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:57:48.688110 (+ 382us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:57:48.688114 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:57:48.688140 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 04:57:48.688579 (+ 439us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:57:48.688589 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:57:48.689884 (+ 1295us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:57:48.689887 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:57:48.692141 (+ 2254us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:57:48.692162 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:57:48.693766 (+ 1604us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:57:48.693776 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:57:48.693915 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:57:48.693944 (+ 29us) client_negotiation.cc:769] Sending connection context
0806 04:57:48.694060 (+ 116us) client_negotiation.cc:240] Negotiation successful
0806 04:57:48.694174 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":510,"mutex_wait_us":1,"thread_start_us":275,"threads_started":1}
I20260806 04:57:48.694746 16531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:57:48.687780 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.96.65:55183 (local address 127.9.96.67:46759)
0806 04:57:48.687975 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0806 04:57:48.687981 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 04:57:48.688380 (+ 399us) server_negotiation.cc:407] Connection header received
0806 04:57:48.688539 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:57:48.688543 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:57:48.688592 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:57:48.688660 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 04:57:48.689152 (+ 492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:57:48.690081 (+ 929us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:57:48.692417 (+ 2336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:57:48.693711 (+ 1294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:57:48.693817 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:57:48.693928 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0806 04:57:48.694376 (+ 448us) server_negotiation.cc:299] Negotiation successful
0806 04:57:48.694501 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260806 04:57:48.694790 16545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:57:48.687669 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.67:46759 (local address 127.9.96.65:55183)
0806 04:57:48.688274 (+ 605us) negotiation.cc:107] Waiting for socket to connect
0806 04:57:48.688296 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0806 04:57:48.688422 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:57:48.688704 (+ 282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:57:48.688710 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:57:48.688737 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 04:57:48.689012 (+ 275us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:57:48.689019 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:57:48.690202 (+ 1183us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:57:48.690207 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:57:48.692273 (+ 2066us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:57:48.692286 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:57:48.693998 (+ 1712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:57:48.694003 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:57:48.694186 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:57:48.694218 (+ 32us) client_negotiation.cc:769] Sending connection context
0806 04:57:48.694319 (+ 101us) client_negotiation.cc:240] Negotiation successful
0806 04:57:48.694440 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":467,"thread_start_us":311,"threads_started":1}
I20260806 04:57:48.695935 16244 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "74a243b846bf4c48a1c4d1bcbb356b01" candidate_uuid: "422c9bcdb6044fb99fdfe5d55d83b545" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a1ce135e8863478183660033cf89cab3" is_pre_election: true
I20260806 04:57:48.695933 16438 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "74a243b846bf4c48a1c4d1bcbb356b01" candidate_uuid: "422c9bcdb6044fb99fdfe5d55d83b545" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c564852e7c874383938f7800a189926a" is_pre_election: true
I20260806 04:57:48.696468 16438 raft_consensus.cc:2454] T 74a243b846bf4c48a1c4d1bcbb356b01 P c564852e7c874383938f7800a189926a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 422c9bcdb6044fb99fdfe5d55d83b545 in term 0.
I20260806 04:57:48.696468 16244 raft_consensus.cc:2454] T 74a243b846bf4c48a1c4d1bcbb356b01 P a1ce135e8863478183660033cf89cab3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 422c9bcdb6044fb99fdfe5d55d83b545 in term 0.
I20260806 04:57:48.697129 15920 leader_election.cc:302] T 74a243b846bf4c48a1c4d1bcbb356b01 P 422c9bcdb6044fb99fdfe5d55d83b545 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 422c9bcdb6044fb99fdfe5d55d83b545, a1ce135e8863478183660033cf89cab3; no voters:
I20260806 04:57:48.697494 16538 raft_consensus.cc:2781] T 74a243b846bf4c48a1c4d1bcbb356b01 P 422c9bcdb6044fb99fdfe5d55d83b545 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 04:57:48.697633 16538 raft_consensus.cc:482] T 74a243b846bf4c48a1c4d1bcbb356b01 P 422c9bcdb6044fb99fdfe5d55d83b545 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 04:57:48.697741 16538 raft_consensus.cc:3037] T 74a243b846bf4c48a1c4d1bcbb356b01 P 422c9bcdb6044fb99fdfe5d55d83b545 [term 0 FOLLOWER]: Advancing to term 1
I20260806 04:57:48.699986 16538 raft_consensus.cc:504] T 74a243b846bf4c48a1c4d1bcbb356b01 P 422c9bcdb6044fb99fdfe5d55d83b545 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "422c9bcdb6044fb99fdfe5d55d83b545" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 39883 } } peers { permanent_uuid: "a1ce135e8863478183660033cf89cab3" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 40759 } } peers { permanent_uuid: "c564852e7c874383938f7800a189926a" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 46759 } }
I20260806 04:57:48.700510 16538 leader_election.cc:288] T 74a243b846bf4c48a1c4d1bcbb356b01 P 422c9bcdb6044fb99fdfe5d55d83b545 [CANDIDATE]: Term 1 election: Requested vote from peers a1ce135e8863478183660033cf89cab3 (127.9.96.66:40759), c564852e7c874383938f7800a189926a (127.9.96.67:46759)
I20260806 04:57:48.700946 16244 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "74a243b846bf4c48a1c4d1bcbb356b01" candidate_uuid: "422c9bcdb6044fb99fdfe5d55d83b545" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a1ce135e8863478183660033cf89cab3"
I20260806 04:57:48.701011 16438 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "74a243b846bf4c48a1c4d1bcbb356b01" candidate_uuid: "422c9bcdb6044fb99fdfe5d55d83b545" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c564852e7c874383938f7800a189926a"
I20260806 04:57:48.701121 16244 raft_consensus.cc:3037] T 74a243b846bf4c48a1c4d1bcbb356b01 P a1ce135e8863478183660033cf89cab3 [term 0 FOLLOWER]: Advancing to term 1
I20260806 04:57:48.701241 16438 raft_consensus.cc:3037] T 74a243b846bf4c48a1c4d1bcbb356b01 P c564852e7c874383938f7800a189926a [term 0 FOLLOWER]: Advancing to term 1
I20260806 04:57:48.703363 16244 raft_consensus.cc:2454] T 74a243b846bf4c48a1c4d1bcbb356b01 P a1ce135e8863478183660033cf89cab3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 422c9bcdb6044fb99fdfe5d55d83b545 in term 1.
I20260806 04:57:48.703424 16438 raft_consensus.cc:2454] T 74a243b846bf4c48a1c4d1bcbb356b01 P c564852e7c874383938f7800a189926a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 422c9bcdb6044fb99fdfe5d55d83b545 in term 1.
I20260806 04:57:48.703774 15920 leader_election.cc:302] T 74a243b846bf4c48a1c4d1bcbb356b01 P 422c9bcdb6044fb99fdfe5d55d83b545 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 422c9bcdb6044fb99fdfe5d55d83b545, a1ce135e8863478183660033cf89cab3; no voters:
I20260806 04:57:48.704072 16538 raft_consensus.cc:2781] T 74a243b846bf4c48a1c4d1bcbb356b01 P 422c9bcdb6044fb99fdfe5d55d83b545 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 04:57:48.704511 16538 raft_consensus.cc:686] T 74a243b846bf4c48a1c4d1bcbb356b01 P 422c9bcdb6044fb99fdfe5d55d83b545 [term 1 LEADER]: Becoming Leader. State: Replica: 422c9bcdb6044fb99fdfe5d55d83b545, State: Running, Role: LEADER
I20260806 04:57:48.705055 16538 consensus_queue.cc:237] T 74a243b846bf4c48a1c4d1bcbb356b01 P 422c9bcdb6044fb99fdfe5d55d83b545 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "422c9bcdb6044fb99fdfe5d55d83b545" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 39883 } } peers { permanent_uuid: "a1ce135e8863478183660033cf89cab3" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 40759 } } peers { permanent_uuid: "c564852e7c874383938f7800a189926a" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 46759 } }
I20260806 04:57:48.709527 15799 catalog_manager.cc:5495] T 74a243b846bf4c48a1c4d1bcbb356b01 P 422c9bcdb6044fb99fdfe5d55d83b545 reported cstate change: term changed from 0 to 1, leader changed from <none> to 422c9bcdb6044fb99fdfe5d55d83b545 (127.9.96.65). New cstate: current_term: 1 leader_uuid: "422c9bcdb6044fb99fdfe5d55d83b545" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "422c9bcdb6044fb99fdfe5d55d83b545" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 39883 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a1ce135e8863478183660033cf89cab3" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 40759 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c564852e7c874383938f7800a189926a" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 46759 } health_report { overall_health: UNKNOWN } } }
I20260806 04:57:48.728664 16526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:57:48.722297 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54956 (local address 127.9.96.65:39883)
0806 04:57:48.722477 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0806 04:57:48.722484 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 04:57:48.722617 (+ 133us) server_negotiation.cc:407] Connection header received
0806 04:57:48.722879 (+ 262us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:57:48.722883 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:57:48.723011 (+ 128us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:57:48.723180 (+ 169us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 04:57:48.723783 (+ 603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:57:48.724710 (+ 927us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:57:48.726049 (+ 1339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:57:48.726467 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:57:48.726601 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:57:48.727240 (+ 639us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 04:57:48.727975 (+ 735us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 04:57:48.728121 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0806 04:57:48.728302 (+ 181us) server_negotiation.cc:299] Negotiation successful
0806 04:57:48.728392 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260806 04:57:48.738205 16438 raft_consensus.cc:1261] T 74a243b846bf4c48a1c4d1bcbb356b01 P c564852e7c874383938f7800a189926a [term 1 FOLLOWER]: Refusing update from remote peer 422c9bcdb6044fb99fdfe5d55d83b545: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 04:57:48.738318 16244 raft_consensus.cc:1261] T 74a243b846bf4c48a1c4d1bcbb356b01 P a1ce135e8863478183660033cf89cab3 [term 1 FOLLOWER]: Refusing update from remote peer 422c9bcdb6044fb99fdfe5d55d83b545: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 04:57:48.739187 16546 consensus_queue.cc:1035] T 74a243b846bf4c48a1c4d1bcbb356b01 P 422c9bcdb6044fb99fdfe5d55d83b545 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c564852e7c874383938f7800a189926a" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 46759 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 04:57:48.739598 16538 consensus_queue.cc:1035] T 74a243b846bf4c48a1c4d1bcbb356b01 P 422c9bcdb6044fb99fdfe5d55d83b545 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a1ce135e8863478183660033cf89cab3" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 40759 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 04:57:48.751883 16554 mvcc.cc:205] Tried to move back new op lower bound from 7315424332742795264 to 7315424332619460608. Current Snapshot: MvccSnapshot[applied={T|T < 7315424332742795264}]
I20260806 04:57:48.763217 15798 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59774:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 04:57:48.763507 15798 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59774:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 04:57:48.766908 15798 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3f6af880361b42bfb3fdfe2fd232c604: Sending DeleteTablet for 3 replicas of tablet 74a243b846bf4c48a1c4d1bcbb356b01
I20260806 04:57:48.768014 15994 tablet_service.cc:1514] Processing DeleteTablet for tablet 74a243b846bf4c48a1c4d1bcbb356b01 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 04:57:48 UTC) from {username='kudu', principal='kudu/127.9.96.126@KRBTEST.COM'} at 127.0.0.1:54948
I20260806 04:57:48.768177 16416 tablet_service.cc:1514] Processing DeleteTablet for tablet 74a243b846bf4c48a1c4d1bcbb356b01 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 04:57:48 UTC) from {username='kudu', principal='kudu/127.9.96.126@KRBTEST.COM'} at 127.0.0.1:36844
I20260806 04:57:48.768389 16213 tablet_service.cc:1514] Processing DeleteTablet for tablet 74a243b846bf4c48a1c4d1bcbb356b01 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 04:57:48 UTC) from {username='kudu', principal='kudu/127.9.96.126@KRBTEST.COM'} at 127.0.0.1:45360
I20260806 04:57:48.770419 9601 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu pid 15896
I20260806 04:57:48.779187 16570 tablet_replica.cc:330] T 74a243b846bf4c48a1c4d1bcbb356b01 P c564852e7c874383938f7800a189926a: stopping tablet replica
I20260806 04:57:48.779812 16570 raft_consensus.cc:2229] T 74a243b846bf4c48a1c4d1bcbb356b01 P c564852e7c874383938f7800a189926a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 04:57:48.780463 16570 raft_consensus.cc:2258] T 74a243b846bf4c48a1c4d1bcbb356b01 P c564852e7c874383938f7800a189926a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 04:57:48.781163 16569 tablet_replica.cc:330] T 74a243b846bf4c48a1c4d1bcbb356b01 P 422c9bcdb6044fb99fdfe5d55d83b545: stopping tablet replica
I20260806 04:57:48.781587 16569 raft_consensus.cc:2229] T 74a243b846bf4c48a1c4d1bcbb356b01 P 422c9bcdb6044fb99fdfe5d55d83b545 [term 1 LEADER]: Raft consensus shutting down.
I20260806 04:57:48.782269 16569 raft_consensus.cc:2258] T 74a243b846bf4c48a1c4d1bcbb356b01 P 422c9bcdb6044fb99fdfe5d55d83b545 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 04:57:48.784881 16569 ts_tablet_manager.cc:1905] T 74a243b846bf4c48a1c4d1bcbb356b01 P 422c9bcdb6044fb99fdfe5d55d83b545: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 04:57:48.785130 16570 ts_tablet_manager.cc:1905] T 74a243b846bf4c48a1c4d1bcbb356b01 P c564852e7c874383938f7800a189926a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 04:57:48.792778 16569 ts_tablet_manager.cc:1918] T 74a243b846bf4c48a1c4d1bcbb356b01 P 422c9bcdb6044fb99fdfe5d55d83b545: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 04:57:48.793051 16569 log.cc:1196] T 74a243b846bf4c48a1c4d1bcbb356b01 P 422c9bcdb6044fb99fdfe5d55d83b545: Deleting WAL directory at /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785992227050689-9601-0/minicluster-data/ts-0/wal/wals/74a243b846bf4c48a1c4d1bcbb356b01
I20260806 04:57:48.793625 16570 ts_tablet_manager.cc:1918] T 74a243b846bf4c48a1c4d1bcbb356b01 P c564852e7c874383938f7800a189926a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 04:57:48.793701 16569 ts_tablet_manager.cc:1939] T 74a243b846bf4c48a1c4d1bcbb356b01 P 422c9bcdb6044fb99fdfe5d55d83b545: Deleting consensus metadata
I20260806 04:57:48.793828 16570 log.cc:1196] T 74a243b846bf4c48a1c4d1bcbb356b01 P c564852e7c874383938f7800a189926a: Deleting WAL directory at /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785992227050689-9601-0/minicluster-data/ts-2/wal/wals/74a243b846bf4c48a1c4d1bcbb356b01
I20260806 04:57:48.794476 16570 ts_tablet_manager.cc:1939] T 74a243b846bf4c48a1c4d1bcbb356b01 P c564852e7c874383938f7800a189926a: Deleting consensus metadata
I20260806 04:57:48.795696 15772 catalog_manager.cc:4841] TS 422c9bcdb6044fb99fdfe5d55d83b545 (127.9.96.65:39883): tablet 74a243b846bf4c48a1c4d1bcbb356b01 (table test-table [id=29e0f2ae980a43d8b81d4ccebb400bb3]) successfully deleted
I20260806 04:57:48.796940 15772 catalog_manager.cc:4841] TS c564852e7c874383938f7800a189926a (127.9.96.67:46759): tablet 74a243b846bf4c48a1c4d1bcbb356b01 (table test-table [id=29e0f2ae980a43d8b81d4ccebb400bb3]) successfully deleted
I20260806 04:57:48.797717 16571 tablet_replica.cc:330] T 74a243b846bf4c48a1c4d1bcbb356b01 P a1ce135e8863478183660033cf89cab3: stopping tablet replica
I20260806 04:57:48.798074 16571 raft_consensus.cc:2229] T 74a243b846bf4c48a1c4d1bcbb356b01 P a1ce135e8863478183660033cf89cab3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 04:57:48.798489 16571 raft_consensus.cc:2258] T 74a243b846bf4c48a1c4d1bcbb356b01 P a1ce135e8863478183660033cf89cab3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 04:57:48.800350 16571 ts_tablet_manager.cc:1905] T 74a243b846bf4c48a1c4d1bcbb356b01 P a1ce135e8863478183660033cf89cab3: Deleting tablet data with delete state TABLET_DATA_DELETED
Aug 06 04:57:48 dist-test-slave-x4qh krb5kdc[15718](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992266, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.96.65@KRBTEST.COM
I20260806 04:57:48.807026 16571 ts_tablet_manager.cc:1918] T 74a243b846bf4c48a1c4d1bcbb356b01 P a1ce135e8863478183660033cf89cab3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 04:57:48.807188 16571 log.cc:1196] T 74a243b846bf4c48a1c4d1bcbb356b01 P a1ce135e8863478183660033cf89cab3: Deleting WAL directory at /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785992227050689-9601-0/minicluster-data/ts-1/wal/wals/74a243b846bf4c48a1c4d1bcbb356b01
I20260806 04:57:48.807720 16571 ts_tablet_manager.cc:1939] T 74a243b846bf4c48a1c4d1bcbb356b01 P a1ce135e8863478183660033cf89cab3: Deleting consensus metadata
I20260806 04:57:48.809235 15772 catalog_manager.cc:4841] TS a1ce135e8863478183660033cf89cab3 (127.9.96.66:40759): tablet 74a243b846bf4c48a1c4d1bcbb356b01 (table test-table [id=29e0f2ae980a43d8b81d4ccebb400bb3]) successfully deleted
I20260806 04:57:48.814481 16526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:57:48.772583 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54962 (local address 127.9.96.65:39883)
0806 04:57:48.772792 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0806 04:57:48.772801 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 04:57:48.796255 (+ 23454us) server_negotiation.cc:407] Connection header received
0806 04:57:48.796492 (+ 237us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:57:48.796503 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:57:48.796587 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:57:48.796711 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:57:48.798545 (+ 1834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:57:48.799792 (+ 1247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:57:48.801525 (+ 1733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:57:48.802243 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:57:48.802397 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:57:48.807546 (+ 5149us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:57:48.807560 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:57:48.807580 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:57:48.807630 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:57:48.811580 (+ 3950us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:57:48.812273 (+ 693us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:57:48.812281 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:57:48.812285 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:57:48.812516 (+ 231us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:57:48.812853 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:57:48.812858 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:57:48.812862 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:57:48.813317 (+ 455us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:57:48.813624 (+ 307us) server_negotiation.cc:1035] Waiting for connection context
0806 04:57:48.813784 (+ 160us) server_negotiation.cc:299] Negotiation successful
0806 04:57:48.814047 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260806 04:57:48.815198 16095 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 04:57:48.914336 9601 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu with pid 15896
I20260806 04:57:48.940631 9601 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu pid 16111
Aug 06 04:57:48 dist-test-slave-x4qh krb5kdc[15718](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992266, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.96.66@KRBTEST.COM
I20260806 04:57:48.955147 16529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:57:48.941446 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45370 (local address 127.9.96.66:40759)
0806 04:57:48.941631 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0806 04:57:48.941635 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 04:57:48.941654 (+ 19us) server_negotiation.cc:407] Connection header received
0806 04:57:48.941758 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:57:48.941761 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:57:48.941853 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:57:48.941940 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:57:48.943110 (+ 1170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:57:48.943988 (+ 878us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:57:48.945190 (+ 1202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:57:48.945574 (+ 384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:57:48.945712 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:57:48.949730 (+ 4018us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:57:48.949742 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:57:48.949757 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:57:48.949786 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:57:48.952454 (+ 2668us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:57:48.953071 (+ 617us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:57:48.953077 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:57:48.953079 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:57:48.953200 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:57:48.953637 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:57:48.953645 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:57:48.953649 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:57:48.954074 (+ 425us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:57:48.954321 (+ 247us) server_negotiation.cc:1035] Waiting for connection context
0806 04:57:48.954650 (+ 329us) server_negotiation.cc:299] Negotiation successful
0806 04:57:48.954855 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260806 04:57:48.955875 16304 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 04:57:49.042846 9601 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu with pid 16111
I20260806 04:57:49.069720 9601 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu pid 16322
Aug 06 04:57:49 dist-test-slave-x4qh krb5kdc[15718](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992266, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.96.67@KRBTEST.COM
I20260806 04:57:49.088493 16531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:57:49.070473 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36852 (local address 127.9.96.67:46759)
0806 04:57:49.070641 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0806 04:57:49.070645 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 04:57:49.070663 (+ 18us) server_negotiation.cc:407] Connection header received
0806 04:57:49.070835 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:57:49.070839 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:57:49.070902 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:57:49.070979 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:57:49.072285 (+ 1306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:57:49.073025 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:57:49.074393 (+ 1368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:57:49.074760 (+ 367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:57:49.074850 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:57:49.083173 (+ 8323us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:57:49.083188 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:57:49.083205 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:57:49.083245 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:57:49.085980 (+ 2735us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:57:49.086560 (+ 580us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:57:49.086567 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:57:49.086569 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:57:49.086700 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:57:49.087061 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:57:49.087064 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:57:49.087065 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:57:49.087492 (+ 427us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:57:49.087702 (+ 210us) server_negotiation.cc:1035] Waiting for connection context
0806 04:57:49.087969 (+ 267us) server_negotiation.cc:299] Negotiation successful
0806 04:57:49.088136 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260806 04:57:49.089138 16504 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 04:57:49.176915 9601 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu with pid 16322
I20260806 04:57:49.200992 9601 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu pid 15739
I20260806 04:57:49.202090 15849 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 04:57:49.308091 9601 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu with pid 15739
2026-08-06T04:57:49Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (4320 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-08-06T04:57:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T04:57:49Z Disabled control of system clock
I20260806 04:57:49.389153 9601 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
/tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785992227050689-9601-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785992227050689-9601-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785992227050689-9601-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785992227050689-9601-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.96.126:42323
--webserver_interface=127.9.96.126
--webserver_port=0
--builtin_ntp_servers=127.9.96.84:37461
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.96.126:42323
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260806 04:57:49.712697 16585 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 04:57:49.712968 16585 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 04:57:49.713066 16585 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 04:57:49.719944 16585 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 04:57:49.720037 16585 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 04:57:49.720075 16585 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 04:57:49.720130 16585 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 04:57:49.720173 16585 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 04:57:49.728860 16585 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.96.84:37461
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785992227050689-9601-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785992227050689-9601-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.96.126:42323
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.96.126:42323
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785992227050689-9601-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.96.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785992227050689-9601-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 03:56:12 UTC on ccycloud.quasar-guijbm.root.comops.site
build id 4569618
ASAN enabled
I20260806 04:57:49.730042 16585 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 04:57:49.731724 16585 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 04:57:49.743543 16590 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:57:49.743512 16593 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:57:49.743582 16591 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:57:49.744416 16585 server_base.cc:1034] running on GCE node
I20260806 04:57:49.745147 16585 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 04:57:49.746409 16585 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 04:57:49.747660 16585 hybrid_clock.cc:648] HybridClock initialized: now 1785992269747604 us; error 50 us; skew 500 ppm
I20260806 04:57:49.747959 16585 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 04:57:49.750101 16585 webserver.cc:460] Webserver started at http://127.9.96.126:44031/ using document root <none> and password file <none>
I20260806 04:57:49.750736 16585 fs_manager.cc:359] Metadata directory not provided
I20260806 04:57:49.750823 16585 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 04:57:49.751116 16585 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 04:57:49.754283 16585 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785992227050689-9601-0/minicluster-data/master-0/data/instance:
uuid: "858ef5fb80d44f9c8ca9be8f11762d94"
format_stamp: "Formatted at 2026-08-06 04:57:49 on dist-test-slave-x4qh"
server_key: "cc045ae859a40d60921b24702352872a"
server_key_iv: "9f267591711cea1c01dc6f88891c14b5"
server_key_version: "encryptionkey@0"
I20260806 04:57:49.755015 16585 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785992227050689-9601-0/minicluster-data/master-0/wal/instance:
uuid: "858ef5fb80d44f9c8ca9be8f11762d94"
format_stamp: "Formatted at 2026-08-06 04:57:49 on dist-test-slave-x4qh"
server_key: "cc045ae859a40d60921b24702352872a"
server_key_iv: "9f267591711cea1c01dc6f88891c14b5"
server_key_version: "encryptionkey@0"
I20260806 04:57:49.759979 16585 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260806 04:57:49.763173 16602 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 04:57:49.764454 16585 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260806 04:57:49.764590 16585 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785992227050689-9601-0/minicluster-data/master-0/data,/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785992227050689-9601-0/minicluster-data/master-0/wal
uuid: "858ef5fb80d44f9c8ca9be8f11762d94"
format_stamp: "Formatted at 2026-08-06 04:57:49 on dist-test-slave-x4qh"
server_key: "cc045ae859a40d60921b24702352872a"
server_key_iv: "9f267591711cea1c01dc6f88891c14b5"
server_key_version: "encryptionkey@0"
I20260806 04:57:49.764767 16585 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785992227050689-9601-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785992227050689-9601-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785992227050689-9601-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 04:57:49.770323 16585 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 04:57:49.770987 16585 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 04:57:49.771232 16585 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 04:57:49.789822 16585 rpc_server.cc:307] RPC server started. Bound to: 127.9.96.126:42323
I20260806 04:57:49.789863 16679 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.96.126:42323 every 8 connection(s)
I20260806 04:57:49.791066 16585 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785992227050689-9601-0/minicluster-data/master-0/data/info.pb
I20260806 04:57:49.794494 16680 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 04:57:49.797642 9601 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu as pid 16585
I20260806 04:57:49.797866 9601 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785992227050689-9601-0/minicluster-data/master-0/wal/instance
I20260806 04:57:49.798535 9601 external_mini_cluster.cc:1428] Setting key e62e70c2738e274ab8310e5a0978ad00
I20260806 04:57:49.806239 16684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:57:49.800418 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42650 (local address 127.9.96.126:42323)
0806 04:57:49.801341 (+ 923us) server_negotiation.cc:206] Beginning negotiation
0806 04:57:49.801351 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 04:57:49.801392 (+ 41us) server_negotiation.cc:407] Connection header received
0806 04:57:49.802836 (+ 1444us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:57:49.802855 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:57:49.803252 (+ 397us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:57:49.803661 (+ 409us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:57:49.804750 (+ 1089us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:57:49.804756 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:57:49.804767 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 04:57:49.804782 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:57:49.804840 (+ 58us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 04:57:49.804854 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:57:49.805031 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0806 04:57:49.805161 (+ 130us) server_negotiation.cc:299] Negotiation successful
0806 04:57:49.805408 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":566,"thread_start_us":354,"threads_started":1}
I20260806 04:57:49.806913 16680 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 858ef5fb80d44f9c8ca9be8f11762d94: Bootstrap starting.
I20260806 04:57:49.810002 16680 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 858ef5fb80d44f9c8ca9be8f11762d94: Neither blocks nor log segments found. Creating new log.
I20260806 04:57:49.811200 16680 log.cc:824] T 00000000000000000000000000000000 P 858ef5fb80d44f9c8ca9be8f11762d94: Log is configured to *not* fsync() on all Append() calls
I20260806 04:57:49.813635 16680 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 858ef5fb80d44f9c8ca9be8f11762d94: No bootstrap required, opened a new log
I20260806 04:57:49.820676 16680 raft_consensus.cc:348] T 00000000000000000000000000000000 P 858ef5fb80d44f9c8ca9be8f11762d94 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "858ef5fb80d44f9c8ca9be8f11762d94" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 42323 } }
I20260806 04:57:49.821017 16680 raft_consensus.cc:374] T 00000000000000000000000000000000 P 858ef5fb80d44f9c8ca9be8f11762d94 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 04:57:49.821136 16680 raft_consensus.cc:729] T 00000000000000000000000000000000 P 858ef5fb80d44f9c8ca9be8f11762d94 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 858ef5fb80d44f9c8ca9be8f11762d94, State: Initialized, Role: FOLLOWER
I20260806 04:57:49.821733 16680 consensus_queue.cc:260] T 00000000000000000000000000000000 P 858ef5fb80d44f9c8ca9be8f11762d94 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "858ef5fb80d44f9c8ca9be8f11762d94" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 42323 } }
I20260806 04:57:49.821972 16680 raft_consensus.cc:388] T 00000000000000000000000000000000 P 858ef5fb80d44f9c8ca9be8f11762d94 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 04:57:49.822124 16680 raft_consensus.cc:482] T 00000000000000000000000000000000 P 858ef5fb80d44f9c8ca9be8f11762d94 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 04:57:49.822293 16680 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 858ef5fb80d44f9c8ca9be8f11762d94 [term 0 FOLLOWER]: Advancing to term 1
I20260806 04:57:49.824680 16680 raft_consensus.cc:504] T 00000000000000000000000000000000 P 858ef5fb80d44f9c8ca9be8f11762d94 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "858ef5fb80d44f9c8ca9be8f11762d94" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 42323 } }
I20260806 04:57:49.825165 16680 leader_election.cc:302] T 00000000000000000000000000000000 P 858ef5fb80d44f9c8ca9be8f11762d94 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 858ef5fb80d44f9c8ca9be8f11762d94; no voters:
I20260806 04:57:49.825773 16680 leader_election.cc:288] T 00000000000000000000000000000000 P 858ef5fb80d44f9c8ca9be8f11762d94 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 04:57:49.825906 16686 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 858ef5fb80d44f9c8ca9be8f11762d94 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 04:57:49.826716 16686 raft_consensus.cc:686] T 00000000000000000000000000000000 P 858ef5fb80d44f9c8ca9be8f11762d94 [term 1 LEADER]: Becoming Leader. State: Replica: 858ef5fb80d44f9c8ca9be8f11762d94, State: Running, Role: LEADER
I20260806 04:57:49.827330 16686 consensus_queue.cc:237] T 00000000000000000000000000000000 P 858ef5fb80d44f9c8ca9be8f11762d94 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "858ef5fb80d44f9c8ca9be8f11762d94" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 42323 } }
I20260806 04:57:49.827939 16680 sys_catalog.cc:564] T 00000000000000000000000000000000 P 858ef5fb80d44f9c8ca9be8f11762d94 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 04:57:49.830019 16687 sys_catalog.cc:455] T 00000000000000000000000000000000 P 858ef5fb80d44f9c8ca9be8f11762d94 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "858ef5fb80d44f9c8ca9be8f11762d94" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "858ef5fb80d44f9c8ca9be8f11762d94" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 42323 } } }
I20260806 04:57:49.830493 16687 sys_catalog.cc:458] T 00000000000000000000000000000000 P 858ef5fb80d44f9c8ca9be8f11762d94 [sys.catalog]: This master's current role is: LEADER
I20260806 04:57:49.832645 16703 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 04:57:49.830497 16690 sys_catalog.cc:455] T 00000000000000000000000000000000 P 858ef5fb80d44f9c8ca9be8f11762d94 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 858ef5fb80d44f9c8ca9be8f11762d94. Latest consensus state: current_term: 1 leader_uuid: "858ef5fb80d44f9c8ca9be8f11762d94" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "858ef5fb80d44f9c8ca9be8f11762d94" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 42323 } } }
I20260806 04:57:49.832882 16690 sys_catalog.cc:458] T 00000000000000000000000000000000 P 858ef5fb80d44f9c8ca9be8f11762d94 [sys.catalog]: This master's current role is: LEADER
I20260806 04:57:49.838243 16703 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 04:57:49.846937 16703 catalog_manager.cc:1347] Generated new cluster ID: 628b802af1d8451a886f7523eb724f23
I20260806 04:57:49.847080 16703 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 04:57:49.858520 16703 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 04:57:49.858911 16703 catalog_manager.cc:1504] Loading token signing keys...
I20260806 04:57:49.865839 16703 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 858ef5fb80d44f9c8ca9be8f11762d94: Generated new TSK 0
I20260806 04:57:49.866505 16703 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 04:57:49.876683 9601 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
/tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785992227050689-9601-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785992227050689-9601-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785992227050689-9601-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785992227050689-9601-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.96.65:0
--local_ip_for_outbound_sockets=127.9.96.65
--webserver_interface=127.9.96.65
--webserver_port=0
--tserver_master_addrs=127.9.96.126:42323
--builtin_ntp_servers=127.9.96.84:37461
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260806 04:57:50.204917 16718 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 04:57:50.205170 16718 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 04:57:50.205330 16718 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 04:57:50.212987 16718 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 04:57:50.213093 16718 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 04:57:50.213243 16718 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.96.65
I20260806 04:57:50.222036 16718 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.96.84:37461
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785992227050689-9601-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785992227050689-9601-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.96.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785992227050689-9601-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.96.65
--webserver_port=0
--tserver_master_addrs=127.9.96.126:42323
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.96.65
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785992227050689-9601-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 03:56:12 UTC on ccycloud.quasar-guijbm.root.comops.site
build id 4569618
ASAN enabled
I20260806 04:57:50.223168 16718 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 04:57:50.224947 16718 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 04:57:50.238153 16729 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:57:50.238250 16732 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:57:50.238299 16734 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:57:50.239027 16718 server_base.cc:1034] running on GCE node
I20260806 04:57:50.239665 16718 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 04:57:50.240517 16718 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 04:57:50.241717 16718 hybrid_clock.cc:648] HybridClock initialized: now 1785992270241677 us; error 31 us; skew 500 ppm
I20260806 04:57:50.241988 16718 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 04:57:50.244477 16718 webserver.cc:460] Webserver started at http://127.9.96.65:41027/ using document root <none> and password file <none>
I20260806 04:57:50.245108 16718 fs_manager.cc:359] Metadata directory not provided
I20260806 04:57:50.245194 16718 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 04:57:50.245462 16718 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 04:57:50.248420 16718 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785992227050689-9601-0/minicluster-data/ts-0/data/instance:
uuid: "e803e485afaa45fc83d7c414dab0b4a0"
format_stamp: "Formatted at 2026-08-06 04:57:50 on dist-test-slave-x4qh"
server_key: "28e35119f37b968f52cd482c59796e1b"
server_key_iv: "4a5063f8e6e974c462a3719f394eb50b"
server_key_version: "encryptionkey@0"
I20260806 04:57:50.249171 16718 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785992227050689-9601-0/minicluster-data/ts-0/wal/instance:
uuid: "e803e485afaa45fc83d7c414dab0b4a0"
format_stamp: "Formatted at 2026-08-06 04:57:50 on dist-test-slave-x4qh"
server_key: "28e35119f37b968f52cd482c59796e1b"
server_key_iv: "4a5063f8e6e974c462a3719f394eb50b"
server_key_version: "encryptionkey@0"
I20260806 04:57:50.254221 16718 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260806 04:57:50.257589 16741 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 04:57:50.258872 16718 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 04:57:50.259038 16718 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785992227050689-9601-0/minicluster-data/ts-0/data,/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785992227050689-9601-0/minicluster-data/ts-0/wal
uuid: "e803e485afaa45fc83d7c414dab0b4a0"
format_stamp: "Formatted at 2026-08-06 04:57:50 on dist-test-slave-x4qh"
server_key: "28e35119f37b968f52cd482c59796e1b"
server_key_iv: "4a5063f8e6e974c462a3719f394eb50b"
server_key_version: "encryptionkey@0"
I20260806 04:57:50.259330 16718 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785992227050689-9601-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785992227050689-9601-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785992227050689-9601-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 04:57:50.265031 16718 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 04:57:50.265727 16718 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 04:57:50.265950 16718 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 04:57:50.266782 16718 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 04:57:50.268286 16718 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 04:57:50.268376 16718 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 04:57:50.268455 16718 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 04:57:50.268509 16718 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 04:57:50.303170 16718 rpc_server.cc:307] RPC server started. Bound to: 127.9.96.65:34811
I20260806 04:57:50.303249 16921 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.96.65:34811 every 8 connection(s)
I20260806 04:57:50.304581 16718 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785992227050689-9601-0/minicluster-data/ts-0/data/info.pb
I20260806 04:57:50.308218 9601 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu as pid 16718
I20260806 04:57:50.308477 9601 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785992227050689-9601-0/minicluster-data/ts-0/wal/instance
I20260806 04:57:50.309327 9601 external_mini_cluster.cc:1428] Setting key 02c97b33d951bca578e7620673534431
I20260806 04:57:50.312050 9601 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
/tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785992227050689-9601-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785992227050689-9601-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785992227050689-9601-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785992227050689-9601-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.96.66:0
--local_ip_for_outbound_sockets=127.9.96.66
--webserver_interface=127.9.96.66
--webserver_port=0
--tserver_master_addrs=127.9.96.126:42323
--builtin_ntp_servers=127.9.96.84:37461
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260806 04:57:50.313400 16924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:57:50.307230 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.96.65:35549 (local address 127.9.96.126:42323)
0806 04:57:50.307682 (+ 452us) server_negotiation.cc:206] Beginning negotiation
0806 04:57:50.307686 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 04:57:50.309274 (+ 1588us) server_negotiation.cc:407] Connection header received
0806 04:57:50.311012 (+ 1738us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:57:50.311017 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:57:50.311089 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:57:50.311207 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:57:50.312635 (+ 1428us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:57:50.312640 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:57:50.312651 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 04:57:50.312667 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:57:50.312710 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 04:57:50.312722 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:57:50.312852 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0806 04:57:50.313041 (+ 189us) server_negotiation.cc:299] Negotiation successful
0806 04:57:50.313109 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":346,"thread_start_us":257,"threads_started":1}
I20260806 04:57:50.314255 16925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:57:50.307946 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:42323 (local address 127.9.96.65:35549)
0806 04:57:50.309100 (+ 1154us) negotiation.cc:107] Waiting for socket to connect
0806 04:57:50.309155 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0806 04:57:50.310742 (+ 1587us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:57:50.311481 (+ 739us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:57:50.311489 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:57:50.311573 (+ 84us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:57:50.312331 (+ 758us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 04:57:50.312334 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:57:50.312386 (+ 52us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 04:57:50.312470 (+ 84us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 04:57:50.312822 (+ 352us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 04:57:50.312833 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 04:57:50.312841 (+ 8us) client_negotiation.cc:769] Sending connection context
0806 04:57:50.313124 (+ 283us) client_negotiation.cc:240] Negotiation successful
0806 04:57:50.313368 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":609,"thread_start_us":377,"threads_started":1}
I20260806 04:57:50.315766 16922 heartbeater.cc:347] Connected to a master server at 127.9.96.126:42323
I20260806 04:57:50.316077 16922 heartbeater.cc:464] Registering TS with master...
I20260806 04:57:50.316603 16922 heartbeater.cc:511] Master 127.9.96.126:42323 requested a full tablet report, sending...
I20260806 04:57:50.318540 16624 ts_manager.cc:194] Registered new tserver with Master: e803e485afaa45fc83d7c414dab0b4a0 (127.9.96.65:34811)
W20260806 04:57:50.639441 16926 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 04:57:50.639703 16926 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 04:57:50.639792 16926 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 04:57:50.647156 16926 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 04:57:50.647271 16926 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 04:57:50.647445 16926 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.96.66
I20260806 04:57:50.656452 16926 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.96.84:37461
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785992227050689-9601-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785992227050689-9601-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.96.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785992227050689-9601-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.96.66
--webserver_port=0
--tserver_master_addrs=127.9.96.126:42323
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.96.66
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785992227050689-9601-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 03:56:12 UTC on ccycloud.quasar-guijbm.root.comops.site
build id 4569618
ASAN enabled
I20260806 04:57:50.657660 16926 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 04:57:50.659333 16926 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 04:57:50.672739 16933 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:57:50.672818 16932 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:57:50.673069 16935 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:57:50.673764 16926 server_base.cc:1034] running on GCE node
I20260806 04:57:50.674356 16926 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 04:57:50.675249 16926 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 04:57:50.676481 16926 hybrid_clock.cc:648] HybridClock initialized: now 1785992270676459 us; error 80 us; skew 500 ppm
I20260806 04:57:50.676808 16926 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 04:57:50.678915 16926 webserver.cc:460] Webserver started at http://127.9.96.66:38661/ using document root <none> and password file <none>
I20260806 04:57:50.679675 16926 fs_manager.cc:359] Metadata directory not provided
I20260806 04:57:50.679814 16926 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 04:57:50.680128 16926 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 04:57:50.682857 16926 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785992227050689-9601-0/minicluster-data/ts-1/data/instance:
uuid: "e99682f977464dbfbfeec67109bfd4c3"
format_stamp: "Formatted at 2026-08-06 04:57:50 on dist-test-slave-x4qh"
server_key: "ee44ff7f0bf55b916680df91996fd80a"
server_key_iv: "8a4f4b1556dad26725197f4b23429a48"
server_key_version: "encryptionkey@0"
I20260806 04:57:50.683691 16926 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785992227050689-9601-0/minicluster-data/ts-1/wal/instance:
uuid: "e99682f977464dbfbfeec67109bfd4c3"
format_stamp: "Formatted at 2026-08-06 04:57:50 on dist-test-slave-x4qh"
server_key: "ee44ff7f0bf55b916680df91996fd80a"
server_key_iv: "8a4f4b1556dad26725197f4b23429a48"
server_key_version: "encryptionkey@0"
I20260806 04:57:50.688637 16926 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.003s
I20260806 04:57:50.692040 16945 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 04:57:50.693284 16926 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 04:57:50.693460 16926 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785992227050689-9601-0/minicluster-data/ts-1/data,/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785992227050689-9601-0/minicluster-data/ts-1/wal
uuid: "e99682f977464dbfbfeec67109bfd4c3"
format_stamp: "Formatted at 2026-08-06 04:57:50 on dist-test-slave-x4qh"
server_key: "ee44ff7f0bf55b916680df91996fd80a"
server_key_iv: "8a4f4b1556dad26725197f4b23429a48"
server_key_version: "encryptionkey@0"
I20260806 04:57:50.693708 16926 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785992227050689-9601-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785992227050689-9601-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785992227050689-9601-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 04:57:50.699415 16926 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 04:57:50.700209 16926 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 04:57:50.700490 16926 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 04:57:50.701380 16926 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 04:57:50.702715 16926 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 04:57:50.702852 16926 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 04:57:50.702989 16926 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 04:57:50.703080 16926 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 04:57:50.736016 16926 rpc_server.cc:307] RPC server started. Bound to: 127.9.96.66:38873
I20260806 04:57:50.736066 17109 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.96.66:38873 every 8 connection(s)
I20260806 04:57:50.737336 16926 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785992227050689-9601-0/minicluster-data/ts-1/data/info.pb
I20260806 04:57:50.744387 9601 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu as pid 16926
I20260806 04:57:50.744619 9601 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785992227050689-9601-0/minicluster-data/ts-1/wal/instance
I20260806 04:57:50.745078 9601 external_mini_cluster.cc:1428] Setting key c46ed55521df71bb4caaf5bbb345f220
I20260806 04:57:50.745577 16924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:57:50.739860 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.96.66:48929 (local address 127.9.96.126:42323)
0806 04:57:50.740031 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0806 04:57:50.740035 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 04:57:50.741357 (+ 1322us) server_negotiation.cc:407] Connection header received
0806 04:57:50.743031 (+ 1674us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:57:50.743036 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:57:50.743112 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:57:50.743226 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:57:50.744813 (+ 1587us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:57:50.744819 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:57:50.744829 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 04:57:50.744848 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:57:50.744882 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 04:57:50.744894 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:57:50.745021 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0806 04:57:50.745219 (+ 198us) server_negotiation.cc:299] Negotiation successful
0806 04:57:50.745292 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260806 04:57:50.746628 17114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:57:50.740303 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:42323 (local address 127.9.96.66:48929)
0806 04:57:50.741191 (+ 888us) negotiation.cc:107] Waiting for socket to connect
0806 04:57:50.741233 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0806 04:57:50.742760 (+ 1527us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:57:50.743538 (+ 778us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:57:50.743550 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:57:50.743685 (+ 135us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:57:50.744408 (+ 723us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 04:57:50.744410 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:57:50.744446 (+ 36us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 04:57:50.744561 (+ 115us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 04:57:50.744993 (+ 432us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 04:57:50.745004 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 04:57:50.745011 (+ 7us) client_negotiation.cc:769] Sending connection context
0806 04:57:50.745307 (+ 296us) client_negotiation.cc:240] Negotiation successful
0806 04:57:50.745541 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":480,"thread_start_us":279,"threads_started":1}
I20260806 04:57:50.747994 17112 heartbeater.cc:347] Connected to a master server at 127.9.96.126:42323
I20260806 04:57:50.748013 9601 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
/tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785992227050689-9601-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785992227050689-9601-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785992227050689-9601-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785992227050689-9601-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.96.67:0
--local_ip_for_outbound_sockets=127.9.96.67
--webserver_interface=127.9.96.67
--webserver_port=0
--tserver_master_addrs=127.9.96.126:42323
--builtin_ntp_servers=127.9.96.84:37461
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260806 04:57:50.748299 17112 heartbeater.cc:464] Registering TS with master...
I20260806 04:57:50.748853 17112 heartbeater.cc:511] Master 127.9.96.126:42323 requested a full tablet report, sending...
I20260806 04:57:50.750234 16624 ts_manager.cc:194] Registered new tserver with Master: e99682f977464dbfbfeec67109bfd4c3 (127.9.96.66:38873)
W20260806 04:57:51.081423 17115 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 04:57:51.081686 17115 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 04:57:51.081772 17115 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 04:57:51.088991 17115 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 04:57:51.089087 17115 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 04:57:51.089200 17115 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.96.67
I20260806 04:57:51.098130 17115 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.96.84:37461
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785992227050689-9601-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785992227050689-9601-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.96.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785992227050689-9601-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.96.67
--webserver_port=0
--tserver_master_addrs=127.9.96.126:42323
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.96.67
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785992227050689-9601-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 03:56:12 UTC on ccycloud.quasar-guijbm.root.comops.site
build id 4569618
ASAN enabled
I20260806 04:57:51.099329 17115 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 04:57:51.101016 17115 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 04:57:51.113862 17127 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:57:51.113940 17123 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:57:51.113897 17122 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:57:51.115139 17115 server_base.cc:1034] running on GCE node
I20260806 04:57:51.115809 17115 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 04:57:51.116721 17115 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 04:57:51.117949 17115 hybrid_clock.cc:648] HybridClock initialized: now 1785992271117897 us; error 66 us; skew 500 ppm
I20260806 04:57:51.118266 17115 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 04:57:51.120502 17115 webserver.cc:460] Webserver started at http://127.9.96.67:36485/ using document root <none> and password file <none>
I20260806 04:57:51.121196 17115 fs_manager.cc:359] Metadata directory not provided
I20260806 04:57:51.121331 17115 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 04:57:51.121646 17115 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 04:57:51.124514 17115 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785992227050689-9601-0/minicluster-data/ts-2/data/instance:
uuid: "ab3de7002443420f97c3fec64af522ec"
format_stamp: "Formatted at 2026-08-06 04:57:51 on dist-test-slave-x4qh"
server_key: "4dc0a04632deb224bf6111c0eb1ce846"
server_key_iv: "96b5b015053d5571952402af519e4e05"
server_key_version: "encryptionkey@0"
I20260806 04:57:51.125305 17115 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785992227050689-9601-0/minicluster-data/ts-2/wal/instance:
uuid: "ab3de7002443420f97c3fec64af522ec"
format_stamp: "Formatted at 2026-08-06 04:57:51 on dist-test-slave-x4qh"
server_key: "4dc0a04632deb224bf6111c0eb1ce846"
server_key_iv: "96b5b015053d5571952402af519e4e05"
server_key_version: "encryptionkey@0"
I20260806 04:57:51.130522 17115 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.004s
I20260806 04:57:51.134076 17136 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 04:57:51.135308 17115 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 04:57:51.135530 17115 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785992227050689-9601-0/minicluster-data/ts-2/data,/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785992227050689-9601-0/minicluster-data/ts-2/wal
uuid: "ab3de7002443420f97c3fec64af522ec"
format_stamp: "Formatted at 2026-08-06 04:57:51 on dist-test-slave-x4qh"
server_key: "4dc0a04632deb224bf6111c0eb1ce846"
server_key_iv: "96b5b015053d5571952402af519e4e05"
server_key_version: "encryptionkey@0"
I20260806 04:57:51.135771 17115 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785992227050689-9601-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785992227050689-9601-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785992227050689-9601-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 04:57:51.141404 17115 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 04:57:51.142179 17115 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 04:57:51.142467 17115 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 04:57:51.143415 17115 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 04:57:51.144932 17115 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 04:57:51.145058 17115 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 04:57:51.145200 17115 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 04:57:51.145335 17115 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.001s
I20260806 04:57:51.179242 17115 rpc_server.cc:307] RPC server started. Bound to: 127.9.96.67:39873
I20260806 04:57:51.179324 17318 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.96.67:39873 every 8 connection(s)
I20260806 04:57:51.180605 17115 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785992227050689-9601-0/minicluster-data/ts-2/data/info.pb
I20260806 04:57:51.188508 16924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:57:51.183350 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.96.67:37695 (local address 127.9.96.126:42323)
0806 04:57:51.183602 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0806 04:57:51.183607 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 04:57:51.184908 (+ 1301us) server_negotiation.cc:407] Connection header received
0806 04:57:51.186513 (+ 1605us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:57:51.186518 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:57:51.186571 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:57:51.186659 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:57:51.187911 (+ 1252us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:57:51.187915 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:57:51.187918 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 04:57:51.187935 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:57:51.187967 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 04:57:51.187979 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:57:51.188084 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0806 04:57:51.188247 (+ 163us) server_negotiation.cc:299] Negotiation successful
0806 04:57:51.188298 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92}
I20260806 04:57:51.189345 17324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:57:51.183871 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:42323 (local address 127.9.96.67:37695)
0806 04:57:51.184762 (+ 891us) negotiation.cc:107] Waiting for socket to connect
0806 04:57:51.184802 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0806 04:57:51.186276 (+ 1474us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:57:51.186852 (+ 576us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:57:51.186868 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:57:51.186951 (+ 83us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:57:51.187624 (+ 673us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 04:57:51.187627 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:57:51.187663 (+ 36us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 04:57:51.187743 (+ 80us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 04:57:51.188070 (+ 327us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 04:57:51.188077 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 04:57:51.188082 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 04:57:51.188289 (+ 207us) client_negotiation.cc:240] Negotiation successful
0806 04:57:51.188472 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":465,"thread_start_us":286,"threads_started":1}
I20260806 04:57:51.190375 9601 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu as pid 17115
I20260806 04:57:51.190573 9601 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785992227050689-9601-0/minicluster-data/ts-2/wal/instance
I20260806 04:57:51.190768 17321 heartbeater.cc:347] Connected to a master server at 127.9.96.126:42323
I20260806 04:57:51.191056 17321 heartbeater.cc:464] Registering TS with master...
I20260806 04:57:51.191111 9601 external_mini_cluster.cc:1428] Setting key 67ea8a6c18f4980e954b3beac136c26c
I20260806 04:57:51.191932 17321 heartbeater.cc:511] Master 127.9.96.126:42323 requested a full tablet report, sending...
I20260806 04:57:51.193399 16624 ts_manager.cc:194] Registered new tserver with Master: ab3de7002443420f97c3fec64af522ec (127.9.96.67:39873)
I20260806 04:57:51.194280 9601 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 04:57:51.200994 16924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:57:51.198829 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35694 (local address 127.9.96.126:42323)
0806 04:57:51.198976 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0806 04:57:51.198981 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 04:57:51.199519 (+ 538us) server_negotiation.cc:407] Connection header received
0806 04:57:51.199628 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:57:51.199631 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:57:51.199677 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:57:51.199752 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:57:51.200563 (+ 811us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:57:51.200567 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:57:51.200570 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 04:57:51.200585 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:57:51.200614 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 04:57:51.200624 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:57:51.200718 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0806 04:57:51.200750 (+ 32us) server_negotiation.cc:299] Negotiation successful
0806 04:57:51.200791 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260806 04:57:51.204800 16623 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35694:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 04:57:51.211539 16623 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 04:57:51.240255 17349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:57:51.224623 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44124 (local address 127.9.96.66:38873)
0806 04:57:51.235258 (+ 10635us) server_negotiation.cc:206] Beginning negotiation
0806 04:57:51.235271 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 04:57:51.235310 (+ 39us) server_negotiation.cc:407] Connection header received
0806 04:57:51.235418 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:57:51.235431 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:57:51.235722 (+ 291us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:57:51.235927 (+ 205us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:57:51.238844 (+ 2917us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:57:51.238854 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:57:51.238931 (+ 77us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 04:57:51.238972 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:57:51.239045 (+ 73us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 04:57:51.239073 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:57:51.239273 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0806 04:57:51.239583 (+ 310us) server_negotiation.cc:299] Negotiation successful
0806 04:57:51.239872 (+ 289us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10432,"thread_start_us":10310,"threads_started":1}
I20260806 04:57:51.240506 17348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:57:51.223998 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.66:38873 (local address 127.0.0.1:44124)
0806 04:57:51.225026 (+ 1028us) negotiation.cc:107] Waiting for socket to connect
0806 04:57:51.225054 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0806 04:57:51.225162 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:57:51.237327 (+ 12165us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:57:51.237339 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:57:51.237449 (+ 110us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:57:51.238427 (+ 978us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 04:57:51.238433 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:57:51.238473 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 04:57:51.238580 (+ 107us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 04:57:51.239324 (+ 744us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 04:57:51.239335 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 04:57:51.239342 (+ 7us) client_negotiation.cc:769] Sending connection context
0806 04:57:51.239598 (+ 256us) client_negotiation.cc:240] Negotiation successful
0806 04:57:51.239715 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":850,"thread_start_us":14750,"threads_started":1}
I20260806 04:57:51.241333 17343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:57:51.223258 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.67:39873 (local address 127.0.0.1:50666)
0806 04:57:51.224087 (+ 829us) negotiation.cc:107] Waiting for socket to connect
0806 04:57:51.224152 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0806 04:57:51.224310 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:57:51.239314 (+ 15004us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:57:51.239320 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:57:51.239345 (+ 25us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:57:51.240129 (+ 784us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 04:57:51.240132 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:57:51.240148 (+ 16us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 04:57:51.240188 (+ 40us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 04:57:51.240636 (+ 448us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 04:57:51.240640 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 04:57:51.240645 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 04:57:51.240777 (+ 132us) client_negotiation.cc:240] Negotiation successful
0806 04:57:51.240828 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":504,"thread_start_us":345,"threads_started":1}
I20260806 04:57:51.241549 17344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:57:51.223605 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50666 (local address 127.9.96.67:39873)
0806 04:57:51.238723 (+ 15118us) server_negotiation.cc:206] Beginning negotiation
0806 04:57:51.238732 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 04:57:51.238771 (+ 39us) server_negotiation.cc:407] Connection header received
0806 04:57:51.238860 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:57:51.238869 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:57:51.239086 (+ 217us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:57:51.239289 (+ 203us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:57:51.240426 (+ 1137us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:57:51.240433 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:57:51.240446 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 04:57:51.240467 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:57:51.240518 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 04:57:51.240537 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:57:51.240720 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0806 04:57:51.240785 (+ 65us) server_negotiation.cc:299] Negotiation successful
0806 04:57:51.240894 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14938,"thread_start_us":14773,"threads_started":1}
I20260806 04:57:51.243086 17347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:57:51.224402 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.65:34811 (local address 127.0.0.1:37662)
0806 04:57:51.239018 (+ 14616us) negotiation.cc:107] Waiting for socket to connect
0806 04:57:51.239042 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0806 04:57:51.239158 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:57:51.240856 (+ 1698us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:57:51.240874 (+ 18us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:57:51.240909 (+ 35us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:57:51.241822 (+ 913us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 04:57:51.241825 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:57:51.241841 (+ 16us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 04:57:51.241896 (+ 55us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 04:57:51.242374 (+ 478us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 04:57:51.242380 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 04:57:51.242385 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 04:57:51.242525 (+ 140us) client_negotiation.cc:240] Negotiation successful
0806 04:57:51.242578 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14484,"thread_start_us":760,"threads_started":1}
I20260806 04:57:51.243577 17350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:57:51.224983 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37662 (local address 127.9.96.65:34811)
0806 04:57:51.237987 (+ 13004us) server_negotiation.cc:206] Beginning negotiation
0806 04:57:51.237999 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 04:57:51.239165 (+ 1166us) server_negotiation.cc:407] Connection header received
0806 04:57:51.240091 (+ 926us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:57:51.240102 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:57:51.240304 (+ 202us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:57:51.240873 (+ 569us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:57:51.242048 (+ 1175us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:57:51.242055 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:57:51.242067 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 04:57:51.242097 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:57:51.242159 (+ 62us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 04:57:51.242179 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:57:51.242309 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0806 04:57:51.243131 (+ 822us) server_negotiation.cc:299] Negotiation successful
0806 04:57:51.243230 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12776,"thread_start_us":13018,"threads_started":1}
I20260806 04:57:51.245543 17020 tablet_service.cc:1467] Processing CreateTablet for tablet 9a84850a9fd54703a7c0f5442c88d213 (DEFAULT_TABLE table=test-table [id=454602496e6b45288b21015d9d6eecaf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 04:57:51.245694 17221 tablet_service.cc:1467] Processing CreateTablet for tablet 9a84850a9fd54703a7c0f5442c88d213 (DEFAULT_TABLE table=test-table [id=454602496e6b45288b21015d9d6eecaf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 04:57:51.246979 17020 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9a84850a9fd54703a7c0f5442c88d213. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 04:57:51.247170 17221 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9a84850a9fd54703a7c0f5442c88d213. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 04:57:51.247184 16818 tablet_service.cc:1467] Processing CreateTablet for tablet 9a84850a9fd54703a7c0f5442c88d213 (DEFAULT_TABLE table=test-table [id=454602496e6b45288b21015d9d6eecaf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 04:57:51.248564 16818 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9a84850a9fd54703a7c0f5442c88d213. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 04:57:51.257520 17351 tablet_bootstrap.cc:492] T 9a84850a9fd54703a7c0f5442c88d213 P e99682f977464dbfbfeec67109bfd4c3: Bootstrap starting.
I20260806 04:57:51.260809 17351 tablet_bootstrap.cc:654] T 9a84850a9fd54703a7c0f5442c88d213 P e99682f977464dbfbfeec67109bfd4c3: Neither blocks nor log segments found. Creating new log.
I20260806 04:57:51.261648 17352 tablet_bootstrap.cc:492] T 9a84850a9fd54703a7c0f5442c88d213 P ab3de7002443420f97c3fec64af522ec: Bootstrap starting.
I20260806 04:57:51.262120 17351 log.cc:824] T 9a84850a9fd54703a7c0f5442c88d213 P e99682f977464dbfbfeec67109bfd4c3: Log is configured to *not* fsync() on all Append() calls
I20260806 04:57:51.265933 17352 tablet_bootstrap.cc:654] T 9a84850a9fd54703a7c0f5442c88d213 P ab3de7002443420f97c3fec64af522ec: Neither blocks nor log segments found. Creating new log.
I20260806 04:57:51.267522 17352 log.cc:824] T 9a84850a9fd54703a7c0f5442c88d213 P ab3de7002443420f97c3fec64af522ec: Log is configured to *not* fsync() on all Append() calls
I20260806 04:57:51.268515 17353 tablet_bootstrap.cc:492] T 9a84850a9fd54703a7c0f5442c88d213 P e803e485afaa45fc83d7c414dab0b4a0: Bootstrap starting.
I20260806 04:57:51.271616 17351 tablet_bootstrap.cc:492] T 9a84850a9fd54703a7c0f5442c88d213 P e99682f977464dbfbfeec67109bfd4c3: No bootstrap required, opened a new log
I20260806 04:57:51.271898 17351 ts_tablet_manager.cc:1397] T 9a84850a9fd54703a7c0f5442c88d213 P e99682f977464dbfbfeec67109bfd4c3: Time spent bootstrapping tablet: real 0.015s user 0.009s sys 0.004s
I20260806 04:57:51.272145 17353 tablet_bootstrap.cc:654] T 9a84850a9fd54703a7c0f5442c88d213 P e803e485afaa45fc83d7c414dab0b4a0: Neither blocks nor log segments found. Creating new log.
I20260806 04:57:51.272275 17352 tablet_bootstrap.cc:492] T 9a84850a9fd54703a7c0f5442c88d213 P ab3de7002443420f97c3fec64af522ec: No bootstrap required, opened a new log
I20260806 04:57:51.272540 17352 ts_tablet_manager.cc:1397] T 9a84850a9fd54703a7c0f5442c88d213 P ab3de7002443420f97c3fec64af522ec: Time spent bootstrapping tablet: real 0.011s user 0.005s sys 0.004s
I20260806 04:57:51.273304 17353 log.cc:824] T 9a84850a9fd54703a7c0f5442c88d213 P e803e485afaa45fc83d7c414dab0b4a0: Log is configured to *not* fsync() on all Append() calls
I20260806 04:57:51.275643 17353 tablet_bootstrap.cc:492] T 9a84850a9fd54703a7c0f5442c88d213 P e803e485afaa45fc83d7c414dab0b4a0: No bootstrap required, opened a new log
I20260806 04:57:51.275894 17353 ts_tablet_manager.cc:1397] T 9a84850a9fd54703a7c0f5442c88d213 P e803e485afaa45fc83d7c414dab0b4a0: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260806 04:57:51.283077 17352 raft_consensus.cc:348] T 9a84850a9fd54703a7c0f5442c88d213 P ab3de7002443420f97c3fec64af522ec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab3de7002443420f97c3fec64af522ec" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 39873 } } peers { permanent_uuid: "e803e485afaa45fc83d7c414dab0b4a0" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 34811 } } peers { permanent_uuid: "e99682f977464dbfbfeec67109bfd4c3" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 38873 } }
I20260806 04:57:51.283077 17351 raft_consensus.cc:348] T 9a84850a9fd54703a7c0f5442c88d213 P e99682f977464dbfbfeec67109bfd4c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab3de7002443420f97c3fec64af522ec" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 39873 } } peers { permanent_uuid: "e803e485afaa45fc83d7c414dab0b4a0" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 34811 } } peers { permanent_uuid: "e99682f977464dbfbfeec67109bfd4c3" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 38873 } }
I20260806 04:57:51.283107 17353 raft_consensus.cc:348] T 9a84850a9fd54703a7c0f5442c88d213 P e803e485afaa45fc83d7c414dab0b4a0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab3de7002443420f97c3fec64af522ec" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 39873 } } peers { permanent_uuid: "e803e485afaa45fc83d7c414dab0b4a0" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 34811 } } peers { permanent_uuid: "e99682f977464dbfbfeec67109bfd4c3" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 38873 } }
I20260806 04:57:51.283569 17351 raft_consensus.cc:374] T 9a84850a9fd54703a7c0f5442c88d213 P e99682f977464dbfbfeec67109bfd4c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 04:57:51.283569 17352 raft_consensus.cc:374] T 9a84850a9fd54703a7c0f5442c88d213 P ab3de7002443420f97c3fec64af522ec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 04:57:51.283577 17353 raft_consensus.cc:374] T 9a84850a9fd54703a7c0f5442c88d213 P e803e485afaa45fc83d7c414dab0b4a0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 04:57:51.283671 17353 raft_consensus.cc:729] T 9a84850a9fd54703a7c0f5442c88d213 P e803e485afaa45fc83d7c414dab0b4a0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e803e485afaa45fc83d7c414dab0b4a0, State: Initialized, Role: FOLLOWER
I20260806 04:57:51.283672 17352 raft_consensus.cc:729] T 9a84850a9fd54703a7c0f5442c88d213 P ab3de7002443420f97c3fec64af522ec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ab3de7002443420f97c3fec64af522ec, State: Initialized, Role: FOLLOWER
I20260806 04:57:51.283674 17351 raft_consensus.cc:729] T 9a84850a9fd54703a7c0f5442c88d213 P e99682f977464dbfbfeec67109bfd4c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e99682f977464dbfbfeec67109bfd4c3, State: Initialized, Role: FOLLOWER
I20260806 04:57:51.284314 17353 consensus_queue.cc:260] T 9a84850a9fd54703a7c0f5442c88d213 P e803e485afaa45fc83d7c414dab0b4a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab3de7002443420f97c3fec64af522ec" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 39873 } } peers { permanent_uuid: "e803e485afaa45fc83d7c414dab0b4a0" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 34811 } } peers { permanent_uuid: "e99682f977464dbfbfeec67109bfd4c3" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 38873 } }
I20260806 04:57:51.284315 17351 consensus_queue.cc:260] T 9a84850a9fd54703a7c0f5442c88d213 P e99682f977464dbfbfeec67109bfd4c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab3de7002443420f97c3fec64af522ec" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 39873 } } peers { permanent_uuid: "e803e485afaa45fc83d7c414dab0b4a0" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 34811 } } peers { permanent_uuid: "e99682f977464dbfbfeec67109bfd4c3" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 38873 } }
I20260806 04:57:51.284302 17352 consensus_queue.cc:260] T 9a84850a9fd54703a7c0f5442c88d213 P ab3de7002443420f97c3fec64af522ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab3de7002443420f97c3fec64af522ec" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 39873 } } peers { permanent_uuid: "e803e485afaa45fc83d7c414dab0b4a0" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 34811 } } peers { permanent_uuid: "e99682f977464dbfbfeec67109bfd4c3" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 38873 } }
I20260806 04:57:51.285450 16922 heartbeater.cc:503] Master 127.9.96.126:42323 was elected leader, sending a full tablet report...
I20260806 04:57:51.286127 17353 ts_tablet_manager.cc:1428] T 9a84850a9fd54703a7c0f5442c88d213 P e803e485afaa45fc83d7c414dab0b4a0: Time spent starting tablet: real 0.010s user 0.010s sys 0.000s
I20260806 04:57:51.287585 17112 heartbeater.cc:503] Master 127.9.96.126:42323 was elected leader, sending a full tablet report...
I20260806 04:57:51.287650 17321 heartbeater.cc:503] Master 127.9.96.126:42323 was elected leader, sending a full tablet report...
I20260806 04:57:51.288125 17351 ts_tablet_manager.cc:1428] T 9a84850a9fd54703a7c0f5442c88d213 P e99682f977464dbfbfeec67109bfd4c3: Time spent starting tablet: real 0.016s user 0.015s sys 0.000s
I20260806 04:57:51.288645 17352 ts_tablet_manager.cc:1428] T 9a84850a9fd54703a7c0f5442c88d213 P ab3de7002443420f97c3fec64af522ec: Time spent starting tablet: real 0.016s user 0.016s sys 0.000s
W20260806 04:57:51.307024 16923 tablet.cc:2369] T 9a84850a9fd54703a7c0f5442c88d213 P e803e485afaa45fc83d7c414dab0b4a0: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 04:57:51.432574 17322 tablet.cc:2369] T 9a84850a9fd54703a7c0f5442c88d213 P ab3de7002443420f97c3fec64af522ec: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 04:57:51.489573 17113 tablet.cc:2369] T 9a84850a9fd54703a7c0f5442c88d213 P e99682f977464dbfbfeec67109bfd4c3: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 04:57:51.533948 17367 raft_consensus.cc:482] T 9a84850a9fd54703a7c0f5442c88d213 P ab3de7002443420f97c3fec64af522ec [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 04:57:51.534235 17367 raft_consensus.cc:504] T 9a84850a9fd54703a7c0f5442c88d213 P ab3de7002443420f97c3fec64af522ec [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab3de7002443420f97c3fec64af522ec" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 39873 } } peers { permanent_uuid: "e803e485afaa45fc83d7c414dab0b4a0" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 34811 } } peers { permanent_uuid: "e99682f977464dbfbfeec67109bfd4c3" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 38873 } }
I20260806 04:57:51.535959 17367 leader_election.cc:288] T 9a84850a9fd54703a7c0f5442c88d213 P ab3de7002443420f97c3fec64af522ec [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e803e485afaa45fc83d7c414dab0b4a0 (127.9.96.65:34811), e99682f977464dbfbfeec67109bfd4c3 (127.9.96.66:38873)
I20260806 04:57:51.538702 17349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:57:51.536460 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.96.67:39177 (local address 127.9.96.66:38873)
0806 04:57:51.536625 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0806 04:57:51.536632 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 04:57:51.536732 (+ 100us) server_negotiation.cc:407] Connection header received
0806 04:57:51.536938 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:57:51.536943 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:57:51.537014 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:57:51.537107 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:57:51.538129 (+ 1022us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:57:51.538133 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:57:51.538136 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 04:57:51.538151 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:57:51.538181 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 04:57:51.538193 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:57:51.538325 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0806 04:57:51.538401 (+ 76us) server_negotiation.cc:299] Negotiation successful
0806 04:57:51.538467 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260806 04:57:51.538751 17324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:57:51.536217 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.66:38873 (local address 127.9.96.67:39177)
0806 04:57:51.536624 (+ 407us) negotiation.cc:107] Waiting for socket to connect
0806 04:57:51.536644 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0806 04:57:51.536776 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:57:51.537180 (+ 404us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:57:51.537187 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:57:51.537212 (+ 25us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:57:51.537895 (+ 683us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 04:57:51.537898 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:57:51.537915 (+ 17us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 04:57:51.537972 (+ 57us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 04:57:51.538299 (+ 327us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 04:57:51.538308 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 04:57:51.538312 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 04:57:51.538397 (+ 85us) client_negotiation.cc:240] Negotiation successful
0806 04:57:51.538441 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":272}
I20260806 04:57:51.539585 17350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:57:51.536409 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.96.67:33941 (local address 127.9.96.65:34811)
0806 04:57:51.536640 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0806 04:57:51.536649 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 04:57:51.537235 (+ 586us) server_negotiation.cc:407] Connection header received
0806 04:57:51.537351 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:57:51.537355 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:57:51.537399 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:57:51.537496 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:57:51.538998 (+ 1502us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:57:51.539006 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:57:51.539010 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 04:57:51.539035 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:57:51.539102 (+ 67us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 04:57:51.539122 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:57:51.539248 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0806 04:57:51.539306 (+ 58us) server_negotiation.cc:299] Negotiation successful
0806 04:57:51.539390 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260806 04:57:51.539880 17368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:57:51.536347 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.65:34811 (local address 127.9.96.67:33941)
0806 04:57:51.537123 (+ 776us) negotiation.cc:107] Waiting for socket to connect
0806 04:57:51.537140 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 04:57:51.537235 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:57:51.537877 (+ 642us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:57:51.537881 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:57:51.537905 (+ 24us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:57:51.538654 (+ 749us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 04:57:51.538657 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:57:51.538672 (+ 15us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 04:57:51.538713 (+ 41us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 04:57:51.539221 (+ 508us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 04:57:51.539224 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 04:57:51.539228 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 04:57:51.539308 (+ 80us) client_negotiation.cc:240] Negotiation successful
0806 04:57:51.539407 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":653,"thread_start_us":414,"threads_started":1}
I20260806 04:57:51.539938 17052 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9a84850a9fd54703a7c0f5442c88d213" candidate_uuid: "ab3de7002443420f97c3fec64af522ec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e99682f977464dbfbfeec67109bfd4c3" is_pre_election: true
I20260806 04:57:51.540555 17052 raft_consensus.cc:2454] T 9a84850a9fd54703a7c0f5442c88d213 P e99682f977464dbfbfeec67109bfd4c3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ab3de7002443420f97c3fec64af522ec in term 0.
I20260806 04:57:51.541018 16851 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9a84850a9fd54703a7c0f5442c88d213" candidate_uuid: "ab3de7002443420f97c3fec64af522ec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e803e485afaa45fc83d7c414dab0b4a0" is_pre_election: true
I20260806 04:57:51.541225 17141 leader_election.cc:302] T 9a84850a9fd54703a7c0f5442c88d213 P ab3de7002443420f97c3fec64af522ec [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ab3de7002443420f97c3fec64af522ec, e99682f977464dbfbfeec67109bfd4c3; no voters:
I20260806 04:57:51.541530 16851 raft_consensus.cc:2454] T 9a84850a9fd54703a7c0f5442c88d213 P e803e485afaa45fc83d7c414dab0b4a0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ab3de7002443420f97c3fec64af522ec in term 0.
I20260806 04:57:51.541632 17367 raft_consensus.cc:2781] T 9a84850a9fd54703a7c0f5442c88d213 P ab3de7002443420f97c3fec64af522ec [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 04:57:51.541805 17367 raft_consensus.cc:482] T 9a84850a9fd54703a7c0f5442c88d213 P ab3de7002443420f97c3fec64af522ec [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 04:57:51.541942 17367 raft_consensus.cc:3037] T 9a84850a9fd54703a7c0f5442c88d213 P ab3de7002443420f97c3fec64af522ec [term 0 FOLLOWER]: Advancing to term 1
I20260806 04:57:51.544234 17367 raft_consensus.cc:504] T 9a84850a9fd54703a7c0f5442c88d213 P ab3de7002443420f97c3fec64af522ec [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab3de7002443420f97c3fec64af522ec" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 39873 } } peers { permanent_uuid: "e803e485afaa45fc83d7c414dab0b4a0" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 34811 } } peers { permanent_uuid: "e99682f977464dbfbfeec67109bfd4c3" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 38873 } }
I20260806 04:57:51.544831 17367 leader_election.cc:288] T 9a84850a9fd54703a7c0f5442c88d213 P ab3de7002443420f97c3fec64af522ec [CANDIDATE]: Term 1 election: Requested vote from peers e803e485afaa45fc83d7c414dab0b4a0 (127.9.96.65:34811), e99682f977464dbfbfeec67109bfd4c3 (127.9.96.66:38873)
I20260806 04:57:51.545190 16851 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9a84850a9fd54703a7c0f5442c88d213" candidate_uuid: "ab3de7002443420f97c3fec64af522ec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e803e485afaa45fc83d7c414dab0b4a0"
I20260806 04:57:51.545315 17052 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9a84850a9fd54703a7c0f5442c88d213" candidate_uuid: "ab3de7002443420f97c3fec64af522ec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e99682f977464dbfbfeec67109bfd4c3"
I20260806 04:57:51.545395 16851 raft_consensus.cc:3037] T 9a84850a9fd54703a7c0f5442c88d213 P e803e485afaa45fc83d7c414dab0b4a0 [term 0 FOLLOWER]: Advancing to term 1
I20260806 04:57:51.545519 17052 raft_consensus.cc:3037] T 9a84850a9fd54703a7c0f5442c88d213 P e99682f977464dbfbfeec67109bfd4c3 [term 0 FOLLOWER]: Advancing to term 1
I20260806 04:57:51.548692 17052 raft_consensus.cc:2454] T 9a84850a9fd54703a7c0f5442c88d213 P e99682f977464dbfbfeec67109bfd4c3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ab3de7002443420f97c3fec64af522ec in term 1.
I20260806 04:57:51.548692 16851 raft_consensus.cc:2454] T 9a84850a9fd54703a7c0f5442c88d213 P e803e485afaa45fc83d7c414dab0b4a0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ab3de7002443420f97c3fec64af522ec in term 1.
I20260806 04:57:51.549064 17141 leader_election.cc:302] T 9a84850a9fd54703a7c0f5442c88d213 P ab3de7002443420f97c3fec64af522ec [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ab3de7002443420f97c3fec64af522ec, e99682f977464dbfbfeec67109bfd4c3; no voters:
I20260806 04:57:51.549322 17367 raft_consensus.cc:2781] T 9a84850a9fd54703a7c0f5442c88d213 P ab3de7002443420f97c3fec64af522ec [term 1 FOLLOWER]: Leader election won for term 1
I20260806 04:57:51.549763 17367 raft_consensus.cc:686] T 9a84850a9fd54703a7c0f5442c88d213 P ab3de7002443420f97c3fec64af522ec [term 1 LEADER]: Becoming Leader. State: Replica: ab3de7002443420f97c3fec64af522ec, State: Running, Role: LEADER
I20260806 04:57:51.550222 17367 consensus_queue.cc:237] T 9a84850a9fd54703a7c0f5442c88d213 P ab3de7002443420f97c3fec64af522ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab3de7002443420f97c3fec64af522ec" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 39873 } } peers { permanent_uuid: "e803e485afaa45fc83d7c414dab0b4a0" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 34811 } } peers { permanent_uuid: "e99682f977464dbfbfeec67109bfd4c3" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 38873 } }
I20260806 04:57:51.554529 16624 catalog_manager.cc:5495] T 9a84850a9fd54703a7c0f5442c88d213 P ab3de7002443420f97c3fec64af522ec reported cstate change: term changed from 0 to 1, leader changed from <none> to ab3de7002443420f97c3fec64af522ec (127.9.96.67). New cstate: current_term: 1 leader_uuid: "ab3de7002443420f97c3fec64af522ec" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab3de7002443420f97c3fec64af522ec" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 39873 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e803e485afaa45fc83d7c414dab0b4a0" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 34811 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e99682f977464dbfbfeec67109bfd4c3" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 38873 } health_report { overall_health: UNKNOWN } } }
I20260806 04:57:51.605633 17344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:57:51.603661 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50676 (local address 127.9.96.67:39873)
0806 04:57:51.603836 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0806 04:57:51.603840 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 04:57:51.603859 (+ 19us) server_negotiation.cc:407] Connection header received
0806 04:57:51.604056 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:57:51.604060 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:57:51.604138 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:57:51.604231 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:57:51.605135 (+ 904us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:57:51.605138 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:57:51.605142 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 04:57:51.605155 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:57:51.605182 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 04:57:51.605193 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:57:51.605290 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0806 04:57:51.605402 (+ 112us) server_negotiation.cc:299] Negotiation successful
0806 04:57:51.605447 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260806 04:57:51.613556 16851 raft_consensus.cc:1261] T 9a84850a9fd54703a7c0f5442c88d213 P e803e485afaa45fc83d7c414dab0b4a0 [term 1 FOLLOWER]: Refusing update from remote peer ab3de7002443420f97c3fec64af522ec: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 04:57:51.613577 17052 raft_consensus.cc:1261] T 9a84850a9fd54703a7c0f5442c88d213 P e99682f977464dbfbfeec67109bfd4c3 [term 1 FOLLOWER]: Refusing update from remote peer ab3de7002443420f97c3fec64af522ec: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 04:57:51.614475 17369 consensus_queue.cc:1035] T 9a84850a9fd54703a7c0f5442c88d213 P ab3de7002443420f97c3fec64af522ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "e99682f977464dbfbfeec67109bfd4c3" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 38873 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 04:57:51.614884 17367 consensus_queue.cc:1035] T 9a84850a9fd54703a7c0f5442c88d213 P ab3de7002443420f97c3fec64af522ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "e803e485afaa45fc83d7c414dab0b4a0" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 34811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 04:57:51.635527 17378 mvcc.cc:205] Tried to move back new op lower bound from 7315424344519475200 to 7315424344272994304. Current Snapshot: MvccSnapshot[applied={T|T < 7315424344519475200}]
I20260806 04:57:51.640475 16624 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:35694:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 04:57:51.640736 16624 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:35694:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 04:57:51.644788 16624 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 858ef5fb80d44f9c8ca9be8f11762d94: Sending DeleteTablet for 3 replicas of tablet 9a84850a9fd54703a7c0f5442c88d213
I20260806 04:57:51.646025 17020 tablet_service.cc:1514] Processing DeleteTablet for tablet 9a84850a9fd54703a7c0f5442c88d213 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 04:57:51 UTC) from {username='slave'} at 127.0.0.1:44124
I20260806 04:57:51.646270 17221 tablet_service.cc:1514] Processing DeleteTablet for tablet 9a84850a9fd54703a7c0f5442c88d213 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 04:57:51 UTC) from {username='slave'} at 127.0.0.1:50666
I20260806 04:57:51.646543 17017 tablet_service.cc:1514] Processing DeleteTablet for tablet 9a84850a9fd54703a7c0f5442c88d213 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 04:57:51 UTC) from {username='slave'} at 127.0.0.1:44124
I20260806 04:57:51.647019 16818 tablet_service.cc:1514] Processing DeleteTablet for tablet 9a84850a9fd54703a7c0f5442c88d213 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 04:57:51 UTC) from {username='slave'} at 127.0.0.1:37662
I20260806 04:57:51.650164 9601 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu pid 16718
I20260806 04:57:51.657121 17395 tablet_replica.cc:330] T 9a84850a9fd54703a7c0f5442c88d213 P e803e485afaa45fc83d7c414dab0b4a0: stopping tablet replica
I20260806 04:57:51.657680 17395 raft_consensus.cc:2229] T 9a84850a9fd54703a7c0f5442c88d213 P e803e485afaa45fc83d7c414dab0b4a0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 04:57:51.658281 17395 raft_consensus.cc:2258] T 9a84850a9fd54703a7c0f5442c88d213 P e803e485afaa45fc83d7c414dab0b4a0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 04:57:51.660576 17395 ts_tablet_manager.cc:1905] T 9a84850a9fd54703a7c0f5442c88d213 P e803e485afaa45fc83d7c414dab0b4a0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 04:57:51.661178 17394 tablet_replica.cc:330] T 9a84850a9fd54703a7c0f5442c88d213 P ab3de7002443420f97c3fec64af522ec: stopping tablet replica
I20260806 04:57:51.661597 17394 raft_consensus.cc:2229] T 9a84850a9fd54703a7c0f5442c88d213 P ab3de7002443420f97c3fec64af522ec [term 1 LEADER]: Raft consensus shutting down.
I20260806 04:57:51.662242 17394 raft_consensus.cc:2258] T 9a84850a9fd54703a7c0f5442c88d213 P ab3de7002443420f97c3fec64af522ec [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 04:57:51.668738 17393 tablet_replica.cc:330] T 9a84850a9fd54703a7c0f5442c88d213 P e99682f977464dbfbfeec67109bfd4c3: stopping tablet replica
I20260806 04:57:51.669188 17393 raft_consensus.cc:2229] T 9a84850a9fd54703a7c0f5442c88d213 P e99682f977464dbfbfeec67109bfd4c3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 04:57:51.669797 17393 raft_consensus.cc:2258] T 9a84850a9fd54703a7c0f5442c88d213 P e99682f977464dbfbfeec67109bfd4c3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 04:57:51.669965 17394 ts_tablet_manager.cc:1905] T 9a84850a9fd54703a7c0f5442c88d213 P ab3de7002443420f97c3fec64af522ec: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 04:57:51.670588 17350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:57:51.652754 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37674 (local address 127.9.96.65:34811)
0806 04:57:51.652992 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0806 04:57:51.653001 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 04:57:51.667892 (+ 14891us) server_negotiation.cc:407] Connection header received
0806 04:57:51.668201 (+ 309us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:57:51.668208 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:57:51.668316 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:57:51.668422 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:57:51.669746 (+ 1324us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:57:51.669752 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:57:51.669757 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 04:57:51.669783 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:57:51.669836 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 04:57:51.669853 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:57:51.669956 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0806 04:57:51.670222 (+ 266us) server_negotiation.cc:299] Negotiation successful
0806 04:57:51.670287 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260806 04:57:51.671731 16916 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 04:57:51.672324 17393 ts_tablet_manager.cc:1905] T 9a84850a9fd54703a7c0f5442c88d213 P e99682f977464dbfbfeec67109bfd4c3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 04:57:51.678840 17394 ts_tablet_manager.cc:1918] T 9a84850a9fd54703a7c0f5442c88d213 P ab3de7002443420f97c3fec64af522ec: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 04:57:51.679019 17394 log.cc:1196] T 9a84850a9fd54703a7c0f5442c88d213 P ab3de7002443420f97c3fec64af522ec: Deleting WAL directory at /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785992227050689-9601-0/minicluster-data/ts-2/wal/wals/9a84850a9fd54703a7c0f5442c88d213
I20260806 04:57:51.679584 17394 ts_tablet_manager.cc:1939] T 9a84850a9fd54703a7c0f5442c88d213 P ab3de7002443420f97c3fec64af522ec: Deleting consensus metadata
I20260806 04:57:51.681283 16603 catalog_manager.cc:4841] TS ab3de7002443420f97c3fec64af522ec (127.9.96.67:39873): tablet 9a84850a9fd54703a7c0f5442c88d213 (table test-table [id=454602496e6b45288b21015d9d6eecaf]) successfully deleted
I20260806 04:57:51.681761 17393 ts_tablet_manager.cc:1918] T 9a84850a9fd54703a7c0f5442c88d213 P e99682f977464dbfbfeec67109bfd4c3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 04:57:51.682077 17393 log.cc:1196] T 9a84850a9fd54703a7c0f5442c88d213 P e99682f977464dbfbfeec67109bfd4c3: Deleting WAL directory at /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785992227050689-9601-0/minicluster-data/ts-1/wal/wals/9a84850a9fd54703a7c0f5442c88d213
I20260806 04:57:51.682761 17393 ts_tablet_manager.cc:1939] T 9a84850a9fd54703a7c0f5442c88d213 P e99682f977464dbfbfeec67109bfd4c3: Deleting consensus metadata
I20260806 04:57:51.684716 16606 catalog_manager.cc:4841] TS e99682f977464dbfbfeec67109bfd4c3 (127.9.96.66:38873): tablet 9a84850a9fd54703a7c0f5442c88d213 (table test-table [id=454602496e6b45288b21015d9d6eecaf]) successfully deleted
W20260806 04:57:51.685562 16606 catalog_manager.cc:4808] TS e99682f977464dbfbfeec67109bfd4c3 (127.9.96.66:38873): delete failed for tablet 9a84850a9fd54703a7c0f5442c88d213 because the tablet was not found. No further retry: Not found: Tablet not found: 9a84850a9fd54703a7c0f5442c88d213
I20260806 04:57:51.795614 17395 ts_tablet_manager.cc:1918] T 9a84850a9fd54703a7c0f5442c88d213 P e803e485afaa45fc83d7c414dab0b4a0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 04:57:51.795931 17395 log.cc:1196] T 9a84850a9fd54703a7c0f5442c88d213 P e803e485afaa45fc83d7c414dab0b4a0: Deleting WAL directory at /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785992227050689-9601-0/minicluster-data/ts-0/wal/wals/9a84850a9fd54703a7c0f5442c88d213
I20260806 04:57:51.796258 9601 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu with pid 16718
W20260806 04:57:51.832813 16603 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.9.96.65:34811 (error 108)
I20260806 04:57:51.833112 9601 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu pid 16926
I20260806 04:57:51.835908 17401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:57:51.834506 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.65:34811 (local address 127.0.0.1:37688)
0806 04:57:51.835506 (+ 1000us) negotiation.cc:107] Waiting for socket to connect
0806 04:57:51.835696 (+ 190us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.96.65:34811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":877,"thread_start_us":731,"threads_started":1}
W20260806 04:57:51.836189 16603 catalog_manager.cc:4584] TS e803e485afaa45fc83d7c414dab0b4a0 (127.9.96.65:34811): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 9a84850a9fd54703a7c0f5442c88d213: Network error: Client connection negotiation failed: client connection to 127.9.96.65:34811: connect: Connection refused (error 111)
I20260806 04:57:51.836699 17349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:57:51.834199 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44140 (local address 127.9.96.66:38873)
0806 04:57:51.834397 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0806 04:57:51.834403 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 04:57:51.834424 (+ 21us) server_negotiation.cc:407] Connection header received
0806 04:57:51.834495 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:57:51.834500 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:57:51.834623 (+ 123us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:57:51.834773 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:57:51.835853 (+ 1080us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:57:51.835858 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:57:51.835863 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 04:57:51.835881 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:57:51.835913 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 04:57:51.835925 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:57:51.836072 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0806 04:57:51.836321 (+ 249us) server_negotiation.cc:299] Negotiation successful
0806 04:57:51.836396 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260806 04:57:51.837405 17103 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 04:57:51.890213 17401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:57:51.889635 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.65:34811 (local address 127.0.0.1:37696)
0806 04:57:51.889908 (+ 273us) negotiation.cc:107] Waiting for socket to connect
0806 04:57:51.890043 (+ 135us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.96.65:34811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102}
I20260806 04:57:51.891535 17401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:57:51.891001 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.65:34811 (local address 127.0.0.1:37704)
0806 04:57:51.891216 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0806 04:57:51.891321 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.96.65:34811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260806 04:57:51.924974 9601 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu with pid 16926
I20260806 04:57:51.950935 9601 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu pid 17115
I20260806 04:57:51.953264 17401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:57:51.952737 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.65:34811 (local address 127.0.0.1:37714)
0806 04:57:51.952981 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0806 04:57:51.953084 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.96.65:34811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100}
I20260806 04:57:51.954429 17344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:57:51.951994 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50682 (local address 127.9.96.67:39873)
0806 04:57:51.952190 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0806 04:57:51.952196 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 04:57:51.952221 (+ 25us) server_negotiation.cc:407] Connection header received
0806 04:57:51.952297 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:57:51.952301 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:57:51.952346 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:57:51.952480 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:57:51.953517 (+ 1037us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:57:51.953524 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:57:51.953529 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 04:57:51.953549 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:57:51.953585 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 04:57:51.953601 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:57:51.953743 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0806 04:57:51.954136 (+ 393us) server_negotiation.cc:299] Negotiation successful
0806 04:57:51.954198 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260806 04:57:51.954530 17401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:57:51.954028 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.65:34811 (local address 127.0.0.1:37724)
0806 04:57:51.954273 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0806 04:57:51.954361 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.96.65:34811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110}
I20260806 04:57:51.955242 17312 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 04:57:52.035355 17401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:57:52.034886 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.65:34811 (local address 127.0.0.1:37736)
0806 04:57:52.035115 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0806 04:57:52.035214 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.96.65:34811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":84}
I20260806 04:57:52.036525 17401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:57:52.036090 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.65:34811 (local address 127.0.0.1:37750)
0806 04:57:52.036297 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0806 04:57:52.036392 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.96.65:34811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96}
I20260806 04:57:52.075276 9601 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu with pid 17115
I20260806 04:57:52.098912 9601 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu pid 16585
I20260806 04:57:52.100284 16675 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 04:57:52.204054 9601 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu with pid 16585
2026-08-06T04:57:52Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (2892 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260806 04:57:52.252799 9601 mini_oidc.cc:152] Starting JWKS server
I20260806 04:57:52.255736 9601 webserver.cc:460] Webserver started at https://127.0.0.1:35613/ using document root <none> and password file <none>
I20260806 04:57:52.256070 9601 mini_oidc.cc:189] Starting OIDC Discovery server
I20260806 04:57:52.256696 9601 webserver.cc:460] Webserver started at http://127.0.0.1:45309/ using document root <none> and password file <none>
2026-08-06T04:57:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T04:57:52Z Disabled control of system clock
I20260806 04:57:52.286935 9601 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
/tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785992227050689-9601-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785992227050689-9601-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785992227050689-9601-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785992227050689-9601-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.96.126:42477
--webserver_interface=127.9.96.126
--webserver_port=0
--builtin_ntp_servers=127.9.96.84:37083
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.96.126:42477
--enable_jwt_token_auth=true
--jwks_url=https://localhost:35613/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785992227050689-9601-0/testchainca.cert with env {}
W20260806 04:57:52.607985 17418 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 04:57:52.608268 17418 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 04:57:52.608356 17418 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 04:57:52.615337 17418 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 04:57:52.615448 17418 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 04:57:52.615520 17418 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 04:57:52.615597 17418 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 04:57:52.615641 17418 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260806 04:57:52.615695 17418 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260806 04:57:52.615736 17418 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:35613/jwks.json
I20260806 04:57:52.625310 17418 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.96.84:37083
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785992227050689-9601-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785992227050689-9601-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.96.126:42477
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.96.126:42477
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:35613/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785992227050689-9601-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.96.126
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785992227050689-9601-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785992227050689-9601-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 03:56:12 UTC on ccycloud.quasar-guijbm.root.comops.site
build id 4569618
ASAN enabled
I20260806 04:57:52.626441 17418 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 04:57:52.627642 17418 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 04:57:52.640249 17430 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:57:52.640249 17431 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:57:52.640386 17435 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:57:52.641165 17418 server_base.cc:1034] running on GCE node
I20260806 04:57:52.641868 17418 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 04:57:52.643164 17418 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 04:57:52.644392 17418 hybrid_clock.cc:648] HybridClock initialized: now 1785992272644336 us; error 37 us; skew 500 ppm
I20260806 04:57:52.644694 17418 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 04:57:52.646900 17418 webserver.cc:460] Webserver started at http://127.9.96.126:40675/ using document root <none> and password file <none>
I20260806 04:57:52.647642 17418 fs_manager.cc:359] Metadata directory not provided
I20260806 04:57:52.647853 17418 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 04:57:52.648177 17418 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 04:57:52.651067 17418 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785992227050689-9601-0/minicluster-data/master-0/data/instance:
uuid: "1447cd44bcec4a4999bc0815c4e8e926"
format_stamp: "Formatted at 2026-08-06 04:57:52 on dist-test-slave-x4qh"
server_key: "587817e4cd4bdc549fdde06a439ce8a9"
server_key_iv: "d3d6f1be1db59e57f9fe2f06690bd30c"
server_key_version: "encryptionkey@0"
I20260806 04:57:52.651863 17418 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785992227050689-9601-0/minicluster-data/master-0/wal/instance:
uuid: "1447cd44bcec4a4999bc0815c4e8e926"
format_stamp: "Formatted at 2026-08-06 04:57:52 on dist-test-slave-x4qh"
server_key: "587817e4cd4bdc549fdde06a439ce8a9"
server_key_iv: "d3d6f1be1db59e57f9fe2f06690bd30c"
server_key_version: "encryptionkey@0"
I20260806 04:57:52.656745 17418 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260806 04:57:52.659989 17443 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 04:57:52.661304 17418 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260806 04:57:52.661494 17418 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785992227050689-9601-0/minicluster-data/master-0/data,/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785992227050689-9601-0/minicluster-data/master-0/wal
uuid: "1447cd44bcec4a4999bc0815c4e8e926"
format_stamp: "Formatted at 2026-08-06 04:57:52 on dist-test-slave-x4qh"
server_key: "587817e4cd4bdc549fdde06a439ce8a9"
server_key_iv: "d3d6f1be1db59e57f9fe2f06690bd30c"
server_key_version: "encryptionkey@0"
I20260806 04:57:52.661701 17418 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785992227050689-9601-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785992227050689-9601-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785992227050689-9601-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 04:57:52.695297 17418 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 04:57:52.696179 17418 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 04:57:52.696439 17418 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 04:57:52.714525 17418 rpc_server.cc:307] RPC server started. Bound to: 127.9.96.126:42477
I20260806 04:57:52.714558 17519 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.96.126:42477 every 8 connection(s)
I20260806 04:57:52.716014 17418 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785992227050689-9601-0/minicluster-data/master-0/data/info.pb
I20260806 04:57:52.719723 17520 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 04:57:52.725417 9601 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu as pid 17418
I20260806 04:57:52.725632 9601 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785992227050689-9601-0/minicluster-data/master-0/wal/instance
I20260806 04:57:52.726452 9601 external_mini_cluster.cc:1428] Setting key 72523dcee761f67eb5f7ca4069b6c283
I20260806 04:57:52.730623 17520 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1447cd44bcec4a4999bc0815c4e8e926: Bootstrap starting.
I20260806 04:57:52.733904 17520 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1447cd44bcec4a4999bc0815c4e8e926: Neither blocks nor log segments found. Creating new log.
I20260806 04:57:52.734869 17520 log.cc:824] T 00000000000000000000000000000000 P 1447cd44bcec4a4999bc0815c4e8e926: Log is configured to *not* fsync() on all Append() calls
I20260806 04:57:52.737713 17520 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1447cd44bcec4a4999bc0815c4e8e926: No bootstrap required, opened a new log
I20260806 04:57:52.740001 17525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:57:52.728559 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47562 (local address 127.9.96.126:42477)
0806 04:57:52.729541 (+ 982us) server_negotiation.cc:206] Beginning negotiation
0806 04:57:52.729551 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 04:57:52.729590 (+ 39us) server_negotiation.cc:407] Connection header received
0806 04:57:52.731254 (+ 1664us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:57:52.731275 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:57:52.731738 (+ 463us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:57:52.732043 (+ 305us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:57:52.732721 (+ 678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:57:52.733987 (+ 1266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:57:52.735405 (+ 1418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:57:52.736072 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:57:52.736319 (+ 247us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:57:52.737589 (+ 1270us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:57:52.737597 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:57:52.737610 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 04:57:52.738240 (+ 630us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:57:52.738318 (+ 78us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 04:57:52.738339 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:57:52.738479 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0806 04:57:52.738696 (+ 217us) server_negotiation.cc:299] Negotiation successful
0806 04:57:52.738991 (+ 295us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":611,"thread_start_us":405,"threads_started":1}
I20260806 04:57:52.743878 9601 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 04:57:52.746027 17520 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1447cd44bcec4a4999bc0815c4e8e926 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1447cd44bcec4a4999bc0815c4e8e926" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 42477 } }
I20260806 04:57:52.746490 17520 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1447cd44bcec4a4999bc0815c4e8e926 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 04:57:52.746599 17520 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1447cd44bcec4a4999bc0815c4e8e926 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1447cd44bcec4a4999bc0815c4e8e926, State: Initialized, Role: FOLLOWER
I20260806 04:57:52.747145 17520 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1447cd44bcec4a4999bc0815c4e8e926 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1447cd44bcec4a4999bc0815c4e8e926" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 42477 } }
I20260806 04:57:52.747484 17520 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1447cd44bcec4a4999bc0815c4e8e926 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 04:57:52.747644 17520 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1447cd44bcec4a4999bc0815c4e8e926 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 04:57:52.747846 17520 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1447cd44bcec4a4999bc0815c4e8e926 [term 0 FOLLOWER]: Advancing to term 1
I20260806 04:57:52.750003 17520 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1447cd44bcec4a4999bc0815c4e8e926 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1447cd44bcec4a4999bc0815c4e8e926" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 42477 } }
I20260806 04:57:52.750519 17520 leader_election.cc:302] T 00000000000000000000000000000000 P 1447cd44bcec4a4999bc0815c4e8e926 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1447cd44bcec4a4999bc0815c4e8e926; no voters:
I20260806 04:57:52.751084 17520 leader_election.cc:288] T 00000000000000000000000000000000 P 1447cd44bcec4a4999bc0815c4e8e926 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 04:57:52.751194 17532 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1447cd44bcec4a4999bc0815c4e8e926 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 04:57:52.752368 17532 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1447cd44bcec4a4999bc0815c4e8e926 [term 1 LEADER]: Becoming Leader. State: Replica: 1447cd44bcec4a4999bc0815c4e8e926, State: Running, Role: LEADER
I20260806 04:57:52.752956 17532 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1447cd44bcec4a4999bc0815c4e8e926 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1447cd44bcec4a4999bc0815c4e8e926" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 42477 } }
I20260806 04:57:52.753332 17520 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1447cd44bcec4a4999bc0815c4e8e926 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 04:57:52.756031 17534 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1447cd44bcec4a4999bc0815c4e8e926 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1447cd44bcec4a4999bc0815c4e8e926. Latest consensus state: current_term: 1 leader_uuid: "1447cd44bcec4a4999bc0815c4e8e926" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1447cd44bcec4a4999bc0815c4e8e926" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 42477 } } }
I20260806 04:57:52.756139 17533 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1447cd44bcec4a4999bc0815c4e8e926 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1447cd44bcec4a4999bc0815c4e8e926" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1447cd44bcec4a4999bc0815c4e8e926" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 42477 } } }
I20260806 04:57:52.756470 17534 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1447cd44bcec4a4999bc0815c4e8e926 [sys.catalog]: This master's current role is: LEADER
I20260806 04:57:52.756554 17533 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1447cd44bcec4a4999bc0815c4e8e926 [sys.catalog]: This master's current role is: LEADER
I20260806 04:57:52.757736 17543 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 04:57:52.764606 17543 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 04:57:52.774158 17543 catalog_manager.cc:1347] Generated new cluster ID: 2fd830c4035344d1aa37b4e8d56374b3
I20260806 04:57:52.774267 17543 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 04:57:52.789705 17543 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 04:57:52.791531 17543 catalog_manager.cc:1504] Loading token signing keys...
I20260806 04:57:52.800976 17543 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1447cd44bcec4a4999bc0815c4e8e926: Generated new TSK 0
I20260806 04:57:52.801972 17543 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260806 04:57:52.851351 17569 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260806 04:57:52.851604 17569 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260806 04:57:52.852751 17525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:57:52.830360 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47568 (local address 127.9.96.126:42477)
0806 04:57:52.830542 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0806 04:57:52.830549 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 04:57:52.830977 (+ 428us) server_negotiation.cc:407] Connection header received
0806 04:57:52.831308 (+ 331us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:57:52.831314 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:57:52.831431 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:57:52.831579 (+ 148us) server_negotiation.cc:226] Negotiated authn=JWT
0806 04:57:52.832069 (+ 490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:57:52.833265 (+ 1196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:57:52.834735 (+ 1470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:57:52.835476 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:57:52.835598 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:57:52.836087 (+ 489us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0806 04:57:52.852130 (+ 16043us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35613/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0806 04:57:52.852467 (+ 337us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:47568: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35613/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":65}
W20260806 04:57:52.852890 17525 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:47568: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35613/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260806 04:57:52.854444 17568 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 04:57:52.830202 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:42477 (local address 127.0.0.1:47568)
0806 04:57:52.830871 (+ 669us) negotiation.cc:107] Waiting for socket to connect
0806 04:57:52.830887 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 04:57:52.831032 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:57:52.831609 (+ 577us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:57:52.831612 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:57:52.831622 (+ 10us) client_negotiation.cc:189] Negotiated authn=JWT
0806 04:57:52.831930 (+ 308us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:57:52.831938 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:57:52.833427 (+ 1489us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:57:52.833433 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:57:52.834593 (+ 1160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:57:52.834604 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:57:52.835635 (+ 1031us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:57:52.835639 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:57:52.835766 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:57:52.835824 (+ 58us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0806 04:57:52.852380 (+ 16556us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35613/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0806 04:57:52.852621 (+ 241us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.9.96.126:42477: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35613/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":481,"thread_start_us":338,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.9.96.126:42477: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35613/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260806 04:57:52.856739 9601 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu pid 17418
I20260806 04:57:52.857929 17514 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 04:57:52.950181 9601 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu with pid 17418
2026-08-06T04:57:52Z chronyd exiting
I20260806 04:57:53.010547 9601 test_util.cc:175] -----------------------------------------------
I20260806 04:57:53.010668 9601 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785992227050689-9601-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (762 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260806 04:57:53.014561 9601 mini_oidc.cc:152] Starting JWKS server
I20260806 04:57:53.016463 9601 webserver.cc:460] Webserver started at https://127.0.0.1:36481/ using document root <none> and password file <none>
I20260806 04:57:53.016705 9601 mini_oidc.cc:189] Starting OIDC Discovery server
I20260806 04:57:53.017341 9601 webserver.cc:460] Webserver started at http://127.0.0.1:42511/ using document root <none> and password file <none>
2026-08-06T04:57:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T04:57:53Z Disabled control of system clock
I20260806 04:57:53.049428 9601 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
/tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785992227050689-9601-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785992227050689-9601-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785992227050689-9601-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785992227050689-9601-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.96.126:38393
--webserver_interface=127.9.96.126
--webserver_port=0
--builtin_ntp_servers=127.9.96.84:38545
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.96.126:38393
--enable_jwt_token_auth=true
--jwks_url=https://localhost:36481/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785992227050689-9601-0/testchainca.cert with env {}
W20260806 04:57:53.374934 17576 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 04:57:53.375306 17576 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 04:57:53.375499 17576 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 04:57:53.382983 17576 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 04:57:53.383142 17576 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 04:57:53.383241 17576 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 04:57:53.383399 17576 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 04:57:53.383496 17576 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260806 04:57:53.383596 17576 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260806 04:57:53.383687 17576 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:36481/jwks.json
I20260806 04:57:53.393671 17576 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.96.84:38545
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785992227050689-9601-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785992227050689-9601-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.96.126:38393
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.96.126:38393
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:36481/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785992227050689-9601-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.96.126
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785992227050689-9601-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785992227050689-9601-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 03:56:12 UTC on ccycloud.quasar-guijbm.root.comops.site
build id 4569618
ASAN enabled
I20260806 04:57:53.395046 17576 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 04:57:53.396428 17576 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 04:57:53.409047 17587 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:57:53.409054 17588 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:57:53.409230 17591 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:57:53.409772 17576 server_base.cc:1034] running on GCE node
I20260806 04:57:53.410565 17576 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 04:57:53.412004 17576 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 04:57:53.413326 17576 hybrid_clock.cc:648] HybridClock initialized: now 1785992273413265 us; error 53 us; skew 500 ppm
I20260806 04:57:53.413728 17576 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 04:57:53.415975 17576 webserver.cc:460] Webserver started at http://127.9.96.126:45259/ using document root <none> and password file <none>
I20260806 04:57:53.416647 17576 fs_manager.cc:359] Metadata directory not provided
I20260806 04:57:53.416774 17576 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 04:57:53.417114 17576 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 04:57:53.420033 17576 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785992227050689-9601-0/minicluster-data/master-0/data/instance:
uuid: "dd3250286c8149d587d84c2c2ea78172"
format_stamp: "Formatted at 2026-08-06 04:57:53 on dist-test-slave-x4qh"
server_key: "618a957b8bf11ee0783e1399e494dc42"
server_key_iv: "a42be0f8f652776bfb5e455169f1745d"
server_key_version: "encryptionkey@0"
I20260806 04:57:53.420816 17576 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785992227050689-9601-0/minicluster-data/master-0/wal/instance:
uuid: "dd3250286c8149d587d84c2c2ea78172"
format_stamp: "Formatted at 2026-08-06 04:57:53 on dist-test-slave-x4qh"
server_key: "618a957b8bf11ee0783e1399e494dc42"
server_key_iv: "a42be0f8f652776bfb5e455169f1745d"
server_key_version: "encryptionkey@0"
I20260806 04:57:53.426988 17576 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.001s
I20260806 04:57:53.430346 17597 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 04:57:53.431751 17576 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 04:57:53.431948 17576 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785992227050689-9601-0/minicluster-data/master-0/data,/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785992227050689-9601-0/minicluster-data/master-0/wal
uuid: "dd3250286c8149d587d84c2c2ea78172"
format_stamp: "Formatted at 2026-08-06 04:57:53 on dist-test-slave-x4qh"
server_key: "618a957b8bf11ee0783e1399e494dc42"
server_key_iv: "a42be0f8f652776bfb5e455169f1745d"
server_key_version: "encryptionkey@0"
I20260806 04:57:53.432168 17576 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785992227050689-9601-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785992227050689-9601-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785992227050689-9601-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 04:57:53.443004 17576 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 04:57:53.443856 17576 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 04:57:53.444162 17576 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 04:57:53.463358 17576 rpc_server.cc:307] RPC server started. Bound to: 127.9.96.126:38393
I20260806 04:57:53.463423 17677 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.96.126:38393 every 8 connection(s)
I20260806 04:57:53.464768 17576 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785992227050689-9601-0/minicluster-data/master-0/data/info.pb
I20260806 04:57:53.468425 17678 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 04:57:53.468631 9601 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu as pid 17576
I20260806 04:57:53.468852 9601 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785992227050689-9601-0/minicluster-data/master-0/wal/instance
I20260806 04:57:53.469497 9601 external_mini_cluster.cc:1428] Setting key 4ba0bf51a1db34ca521439b3cebef668
I20260806 04:57:53.483287 17678 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dd3250286c8149d587d84c2c2ea78172: Bootstrap starting.
I20260806 04:57:53.485289 17685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:57:53.471443 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42788 (local address 127.9.96.126:38393)
0806 04:57:53.472525 (+ 1082us) server_negotiation.cc:206] Beginning negotiation
0806 04:57:53.472539 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0806 04:57:53.472606 (+ 67us) server_negotiation.cc:407] Connection header received
0806 04:57:53.475335 (+ 2729us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:57:53.475364 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:57:53.475988 (+ 624us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:57:53.476439 (+ 451us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:57:53.476980 (+ 541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:57:53.478822 (+ 1842us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:57:53.480159 (+ 1337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:57:53.481220 (+ 1061us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:57:53.481401 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:57:53.482482 (+ 1081us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:57:53.482492 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:57:53.482508 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 04:57:53.483175 (+ 667us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:57:53.483306 (+ 131us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 04:57:53.483337 (+ 31us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:57:53.483482 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0806 04:57:53.483707 (+ 225us) server_negotiation.cc:299] Negotiation successful
0806 04:57:53.484007 (+ 300us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":561,"thread_start_us":326,"threads_started":1}
I20260806 04:57:53.487174 17678 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P dd3250286c8149d587d84c2c2ea78172: Neither blocks nor log segments found. Creating new log.
I20260806 04:57:53.488502 17678 log.cc:824] T 00000000000000000000000000000000 P dd3250286c8149d587d84c2c2ea78172: Log is configured to *not* fsync() on all Append() calls
I20260806 04:57:53.489200 9601 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 04:57:53.491628 17678 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dd3250286c8149d587d84c2c2ea78172: No bootstrap required, opened a new log
I20260806 04:57:53.498754 17678 raft_consensus.cc:348] T 00000000000000000000000000000000 P dd3250286c8149d587d84c2c2ea78172 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd3250286c8149d587d84c2c2ea78172" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 38393 } }
I20260806 04:57:53.499181 17678 raft_consensus.cc:374] T 00000000000000000000000000000000 P dd3250286c8149d587d84c2c2ea78172 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 04:57:53.499310 17678 raft_consensus.cc:729] T 00000000000000000000000000000000 P dd3250286c8149d587d84c2c2ea78172 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dd3250286c8149d587d84c2c2ea78172, State: Initialized, Role: FOLLOWER
I20260806 04:57:53.499904 17678 consensus_queue.cc:260] T 00000000000000000000000000000000 P dd3250286c8149d587d84c2c2ea78172 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd3250286c8149d587d84c2c2ea78172" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 38393 } }
I20260806 04:57:53.500154 17678 raft_consensus.cc:388] T 00000000000000000000000000000000 P dd3250286c8149d587d84c2c2ea78172 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 04:57:53.500317 17678 raft_consensus.cc:482] T 00000000000000000000000000000000 P dd3250286c8149d587d84c2c2ea78172 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 04:57:53.500478 17678 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dd3250286c8149d587d84c2c2ea78172 [term 0 FOLLOWER]: Advancing to term 1
I20260806 04:57:53.502488 17678 raft_consensus.cc:504] T 00000000000000000000000000000000 P dd3250286c8149d587d84c2c2ea78172 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd3250286c8149d587d84c2c2ea78172" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 38393 } }
I20260806 04:57:53.502942 17678 leader_election.cc:302] T 00000000000000000000000000000000 P dd3250286c8149d587d84c2c2ea78172 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: dd3250286c8149d587d84c2c2ea78172; no voters:
I20260806 04:57:53.503597 17678 leader_election.cc:288] T 00000000000000000000000000000000 P dd3250286c8149d587d84c2c2ea78172 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 04:57:53.503742 17687 raft_consensus.cc:2781] T 00000000000000000000000000000000 P dd3250286c8149d587d84c2c2ea78172 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 04:57:53.504742 17687 raft_consensus.cc:686] T 00000000000000000000000000000000 P dd3250286c8149d587d84c2c2ea78172 [term 1 LEADER]: Becoming Leader. State: Replica: dd3250286c8149d587d84c2c2ea78172, State: Running, Role: LEADER
I20260806 04:57:53.505333 17687 consensus_queue.cc:237] T 00000000000000000000000000000000 P dd3250286c8149d587d84c2c2ea78172 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd3250286c8149d587d84c2c2ea78172" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 38393 } }
I20260806 04:57:53.505932 17678 sys_catalog.cc:564] T 00000000000000000000000000000000 P dd3250286c8149d587d84c2c2ea78172 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 04:57:53.508671 17688 sys_catalog.cc:455] T 00000000000000000000000000000000 P dd3250286c8149d587d84c2c2ea78172 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "dd3250286c8149d587d84c2c2ea78172" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd3250286c8149d587d84c2c2ea78172" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 38393 } } }
I20260806 04:57:53.509073 17688 sys_catalog.cc:458] T 00000000000000000000000000000000 P dd3250286c8149d587d84c2c2ea78172 [sys.catalog]: This master's current role is: LEADER
I20260806 04:57:53.509516 17689 sys_catalog.cc:455] T 00000000000000000000000000000000 P dd3250286c8149d587d84c2c2ea78172 [sys.catalog]: SysCatalogTable state changed. Reason: New leader dd3250286c8149d587d84c2c2ea78172. Latest consensus state: current_term: 1 leader_uuid: "dd3250286c8149d587d84c2c2ea78172" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd3250286c8149d587d84c2c2ea78172" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 38393 } } }
I20260806 04:57:53.510021 17689 sys_catalog.cc:458] T 00000000000000000000000000000000 P dd3250286c8149d587d84c2c2ea78172 [sys.catalog]: This master's current role is: LEADER
I20260806 04:57:53.518535 17694 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 04:57:53.524340 17694 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 04:57:53.534109 17694 catalog_manager.cc:1347] Generated new cluster ID: 8103f41bac304709bfa0915505fe03f7
I20260806 04:57:53.534235 17694 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 04:57:53.545313 17694 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 04:57:53.546577 17694 catalog_manager.cc:1504] Loading token signing keys...
I20260806 04:57:53.552642 17694 catalog_manager.cc:5868] T 00000000000000000000000000000000 P dd3250286c8149d587d84c2c2ea78172: Generated new TSK 0
I20260806 04:57:53.553367 17694 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260806 04:57:53.586691 17733 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260806 04:57:53.586829 17733 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
W20260806 04:57:53.588150 17731 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 04:57:53.567934 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:38393 (local address 127.0.0.1:42792)
0806 04:57:53.568615 (+ 681us) negotiation.cc:107] Waiting for socket to connect
0806 04:57:53.568629 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 04:57:53.568748 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:57:53.569175 (+ 427us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:57:53.569178 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:57:53.569187 (+ 9us) client_negotiation.cc:189] Negotiated authn=JWT
0806 04:57:53.569452 (+ 265us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:57:53.569459 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:57:53.570902 (+ 1443us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:57:53.570906 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:57:53.572036 (+ 1130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:57:53.572048 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:57:53.572993 (+ 945us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:57:53.572997 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:57:53.573127 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:57:53.573155 (+ 28us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0806 04:57:53.587674 (+ 14519us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36481/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0806 04:57:53.587873 (+ 199us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.9.96.126:38393: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36481/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":568,"thread_start_us":398,"threads_started":1}
I20260806 04:57:53.588152 17685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:57:53.568050 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42792 (local address 127.9.96.126:38393)
0806 04:57:53.568211 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0806 04:57:53.568216 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 04:57:53.568727 (+ 511us) server_negotiation.cc:407] Connection header received
0806 04:57:53.568887 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:57:53.568893 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:57:53.568993 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:57:53.569163 (+ 170us) server_negotiation.cc:226] Negotiated authn=JWT
0806 04:57:53.569611 (+ 448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:57:53.570766 (+ 1155us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:57:53.572171 (+ 1405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:57:53.572842 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:57:53.572977 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:57:53.573417 (+ 440us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0806 04:57:53.587435 (+ 14018us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36481/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0806 04:57:53.587834 (+ 399us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:42792: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36481/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":54}
W20260806 04:57:53.588294 17685 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:42792: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36481/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
I20260806 04:57:53.589722 9601 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu pid 17576
I20260806 04:57:53.590987 17673 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 04:57:53.680266 9601 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu with pid 17576
2026-08-06T04:57:53Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (728 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260806 04:57:53.744622 9601 mini_oidc.cc:152] Starting JWKS server
I20260806 04:57:53.747189 9601 webserver.cc:460] Webserver started at https://127.0.0.1:33261/ using document root <none> and password file <none>
I20260806 04:57:53.747494 9601 mini_oidc.cc:189] Starting OIDC Discovery server
I20260806 04:57:53.748255 9601 webserver.cc:460] Webserver started at http://127.0.0.1:34583/ using document root <none> and password file <none>
2026-08-06T04:57:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T04:57:53Z Disabled control of system clock
I20260806 04:57:53.780335 9601 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
/tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785992227050689-9601-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785992227050689-9601-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785992227050689-9601-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785992227050689-9601-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.96.126:44659
--webserver_interface=127.9.96.126
--webserver_port=0
--builtin_ntp_servers=127.9.96.84:41029
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.96.126:44659
--enable_jwt_token_auth=true
--jwks_url=https://localhost:33261/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260806 04:57:54.105968 17742 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 04:57:54.106273 17742 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 04:57:54.106439 17742 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 04:57:54.113659 17742 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 04:57:54.113762 17742 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 04:57:54.113838 17742 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 04:57:54.113945 17742 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 04:57:54.114032 17742 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260806 04:57:54.114147 17742 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260806 04:57:54.114231 17742 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:33261/jwks.json
I20260806 04:57:54.123546 17742 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.96.84:41029
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785992227050689-9601-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785992227050689-9601-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.96.126:44659
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.96.126:44659
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:33261/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785992227050689-9601-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.96.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785992227050689-9601-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 03:56:12 UTC on ccycloud.quasar-guijbm.root.comops.site
build id 4569618
ASAN enabled
I20260806 04:57:54.125104 17742 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 04:57:54.126921 17742 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 04:57:54.138585 17757 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:57:54.138782 17756 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:57:54.138892 17763 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:57:54.139968 17742 server_base.cc:1034] running on GCE node
I20260806 04:57:54.140780 17742 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 04:57:54.142138 17742 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 04:57:54.143410 17742 hybrid_clock.cc:648] HybridClock initialized: now 1785992274143357 us; error 67 us; skew 500 ppm
I20260806 04:57:54.143777 17742 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 04:57:54.150095 17742 webserver.cc:460] Webserver started at http://127.9.96.126:38429/ using document root <none> and password file <none>
I20260806 04:57:54.150979 17742 fs_manager.cc:359] Metadata directory not provided
I20260806 04:57:54.151116 17742 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 04:57:54.151571 17742 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 04:57:54.154560 17742 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785992227050689-9601-0/minicluster-data/master-0/data/instance:
uuid: "ea64c28245bc47cf9e670f6693550d8d"
format_stamp: "Formatted at 2026-08-06 04:57:54 on dist-test-slave-x4qh"
server_key: "10d38537638cdc54d469ad76a46c14b8"
server_key_iv: "90b46cad601a8263776cbdceef5e85f0"
server_key_version: "encryptionkey@0"
I20260806 04:57:54.155516 17742 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785992227050689-9601-0/minicluster-data/master-0/wal/instance:
uuid: "ea64c28245bc47cf9e670f6693550d8d"
format_stamp: "Formatted at 2026-08-06 04:57:54 on dist-test-slave-x4qh"
server_key: "10d38537638cdc54d469ad76a46c14b8"
server_key_iv: "90b46cad601a8263776cbdceef5e85f0"
server_key_version: "encryptionkey@0"
I20260806 04:57:54.160693 17742 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260806 04:57:54.164130 17775 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 04:57:54.165584 17742 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 04:57:54.165768 17742 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785992227050689-9601-0/minicluster-data/master-0/data,/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785992227050689-9601-0/minicluster-data/master-0/wal
uuid: "ea64c28245bc47cf9e670f6693550d8d"
format_stamp: "Formatted at 2026-08-06 04:57:54 on dist-test-slave-x4qh"
server_key: "10d38537638cdc54d469ad76a46c14b8"
server_key_iv: "90b46cad601a8263776cbdceef5e85f0"
server_key_version: "encryptionkey@0"
I20260806 04:57:54.165989 17742 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785992227050689-9601-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785992227050689-9601-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785992227050689-9601-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 04:57:54.190263 17742 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 04:57:54.191205 17742 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 04:57:54.191545 17742 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 04:57:54.209770 17742 rpc_server.cc:307] RPC server started. Bound to: 127.9.96.126:44659
I20260806 04:57:54.209744 17853 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.96.126:44659 every 8 connection(s)
I20260806 04:57:54.211071 17742 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785992227050689-9601-0/minicluster-data/master-0/data/info.pb
I20260806 04:57:54.214903 17854 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 04:57:54.220175 9601 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu as pid 17742
I20260806 04:57:54.220383 9601 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785992227050689-9601-0/minicluster-data/master-0/wal/instance
I20260806 04:57:54.221037 9601 external_mini_cluster.cc:1428] Setting key 3af9af1d49a6f67efe43875c8e463e92
I20260806 04:57:54.225870 17854 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ea64c28245bc47cf9e670f6693550d8d: Bootstrap starting.
I20260806 04:57:54.228952 17854 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ea64c28245bc47cf9e670f6693550d8d: Neither blocks nor log segments found. Creating new log.
I20260806 04:57:54.229935 17854 log.cc:824] T 00000000000000000000000000000000 P ea64c28245bc47cf9e670f6693550d8d: Log is configured to *not* fsync() on all Append() calls
I20260806 04:57:54.232857 17854 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ea64c28245bc47cf9e670f6693550d8d: No bootstrap required, opened a new log
I20260806 04:57:54.234210 17858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:57:54.223077 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50036 (local address 127.9.96.126:44659)
0806 04:57:54.224070 (+ 993us) server_negotiation.cc:206] Beginning negotiation
0806 04:57:54.224080 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 04:57:54.224119 (+ 39us) server_negotiation.cc:407] Connection header received
0806 04:57:54.225769 (+ 1650us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:57:54.225790 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:57:54.226268 (+ 478us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:57:54.226635 (+ 367us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:57:54.227182 (+ 547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:57:54.228536 (+ 1354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:57:54.229885 (+ 1349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:57:54.230620 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:57:54.230859 (+ 239us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:57:54.232029 (+ 1170us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:57:54.232036 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:57:54.232049 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 04:57:54.232564 (+ 515us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:57:54.232634 (+ 70us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 04:57:54.232654 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:57:54.232789 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0806 04:57:54.233023 (+ 234us) server_negotiation.cc:299] Negotiation successful
0806 04:57:54.233332 (+ 309us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":585,"thread_start_us":343,"threads_started":1}
I20260806 04:57:54.237895 9601 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 04:57:54.240825 17854 raft_consensus.cc:348] T 00000000000000000000000000000000 P ea64c28245bc47cf9e670f6693550d8d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea64c28245bc47cf9e670f6693550d8d" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 44659 } }
I20260806 04:57:54.241264 17854 raft_consensus.cc:374] T 00000000000000000000000000000000 P ea64c28245bc47cf9e670f6693550d8d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 04:57:54.241350 17854 raft_consensus.cc:729] T 00000000000000000000000000000000 P ea64c28245bc47cf9e670f6693550d8d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ea64c28245bc47cf9e670f6693550d8d, State: Initialized, Role: FOLLOWER
I20260806 04:57:54.241819 17854 consensus_queue.cc:260] T 00000000000000000000000000000000 P ea64c28245bc47cf9e670f6693550d8d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea64c28245bc47cf9e670f6693550d8d" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 44659 } }
I20260806 04:57:54.242064 17854 raft_consensus.cc:388] T 00000000000000000000000000000000 P ea64c28245bc47cf9e670f6693550d8d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 04:57:54.242156 17854 raft_consensus.cc:482] T 00000000000000000000000000000000 P ea64c28245bc47cf9e670f6693550d8d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 04:57:54.242267 17854 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ea64c28245bc47cf9e670f6693550d8d [term 0 FOLLOWER]: Advancing to term 1
I20260806 04:57:54.244675 17854 raft_consensus.cc:504] T 00000000000000000000000000000000 P ea64c28245bc47cf9e670f6693550d8d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea64c28245bc47cf9e670f6693550d8d" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 44659 } }
I20260806 04:57:54.245108 17854 leader_election.cc:302] T 00000000000000000000000000000000 P ea64c28245bc47cf9e670f6693550d8d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ea64c28245bc47cf9e670f6693550d8d; no voters:
I20260806 04:57:54.245586 17854 leader_election.cc:288] T 00000000000000000000000000000000 P ea64c28245bc47cf9e670f6693550d8d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 04:57:54.245748 17860 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ea64c28245bc47cf9e670f6693550d8d [term 1 FOLLOWER]: Leader election won for term 1
I20260806 04:57:54.246480 17860 raft_consensus.cc:686] T 00000000000000000000000000000000 P ea64c28245bc47cf9e670f6693550d8d [term 1 LEADER]: Becoming Leader. State: Replica: ea64c28245bc47cf9e670f6693550d8d, State: Running, Role: LEADER
I20260806 04:57:54.247105 17860 consensus_queue.cc:237] T 00000000000000000000000000000000 P ea64c28245bc47cf9e670f6693550d8d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea64c28245bc47cf9e670f6693550d8d" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 44659 } }
I20260806 04:57:54.247730 17854 sys_catalog.cc:564] T 00000000000000000000000000000000 P ea64c28245bc47cf9e670f6693550d8d [sys.catalog]: configured and running, proceeding with master startup.
I20260806 04:57:54.250003 17862 sys_catalog.cc:455] T 00000000000000000000000000000000 P ea64c28245bc47cf9e670f6693550d8d [sys.catalog]: SysCatalogTable state changed. Reason: New leader ea64c28245bc47cf9e670f6693550d8d. Latest consensus state: current_term: 1 leader_uuid: "ea64c28245bc47cf9e670f6693550d8d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea64c28245bc47cf9e670f6693550d8d" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 44659 } } }
I20260806 04:57:54.250144 17861 sys_catalog.cc:455] T 00000000000000000000000000000000 P ea64c28245bc47cf9e670f6693550d8d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ea64c28245bc47cf9e670f6693550d8d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea64c28245bc47cf9e670f6693550d8d" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 44659 } } }
I20260806 04:57:54.250419 17862 sys_catalog.cc:458] T 00000000000000000000000000000000 P ea64c28245bc47cf9e670f6693550d8d [sys.catalog]: This master's current role is: LEADER
I20260806 04:57:54.250468 17861 sys_catalog.cc:458] T 00000000000000000000000000000000 P ea64c28245bc47cf9e670f6693550d8d [sys.catalog]: This master's current role is: LEADER
I20260806 04:57:54.256923 17870 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 04:57:54.261575 17870 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 04:57:54.270339 17870 catalog_manager.cc:1347] Generated new cluster ID: afcc8c088df24939a9ecd8c93986ddfe
I20260806 04:57:54.270468 17870 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 04:57:54.289220 17870 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 04:57:54.290530 17870 catalog_manager.cc:1504] Loading token signing keys...
I20260806 04:57:54.302739 17870 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ea64c28245bc47cf9e670f6693550d8d: Generated new TSK 0
I20260806 04:57:54.303673 17870 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 04:57:54.323658 17858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:57:54.315480 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50038 (local address 127.9.96.126:44659)
0806 04:57:54.315739 (+ 259us) server_negotiation.cc:206] Beginning negotiation
0806 04:57:54.315752 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 04:57:54.316347 (+ 595us) server_negotiation.cc:407] Connection header received
0806 04:57:54.316604 (+ 257us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:57:54.316608 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:57:54.316682 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:57:54.316864 (+ 182us) server_negotiation.cc:226] Negotiated authn=JWT
0806 04:57:54.317333 (+ 469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:57:54.318093 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:57:54.319592 (+ 1499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:57:54.320051 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:57:54.320205 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:57:54.320693 (+ 488us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0806 04:57:54.323000 (+ 2307us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33261/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0806 04:57:54.323336 (+ 336us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:50038: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33261/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":113}
W20260806 04:57:54.323781 17858 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:50038: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33261/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260806 04:57:54.325914 17895 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 04:57:54.315260 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:44659 (local address 127.0.0.1:50038)
0806 04:57:54.316223 (+ 963us) negotiation.cc:107] Waiting for socket to connect
0806 04:57:54.316250 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0806 04:57:54.316428 (+ 178us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:57:54.316875 (+ 447us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:57:54.316880 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:57:54.316891 (+ 11us) client_negotiation.cc:189] Negotiated authn=JWT
0806 04:57:54.317179 (+ 288us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:57:54.317188 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:57:54.318242 (+ 1054us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:57:54.318247 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:57:54.319413 (+ 1166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:57:54.319431 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:57:54.320252 (+ 821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:57:54.320259 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:57:54.320415 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:57:54.320448 (+ 33us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0806 04:57:54.323217 (+ 2769us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33261/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0806 04:57:54.325438 (+ 2221us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.9.96.126:44659: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33261/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":805,"mutex_wait_us":1898,"thread_start_us":641,"threads_started":1}
I20260806 04:57:54.326890 9601 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu pid 17742
I20260806 04:57:54.327895 17844 generic_service.cc:196] Checking for leaks (request via RPC)
E20260806 04:57:54.336337 17896 webserver.cc:533] Webserver: sslize failed:
E20260806 04:57:54.336426 17896 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260806 04:57:54.419167 9601 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu with pid 17742
2026-08-06T04:57:54Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (739 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-08-06T04:57:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T04:57:54Z Disabled control of system clock
I20260806 04:57:54.514825 9601 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
/tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785992227050689-9601-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785992227050689-9601-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785992227050689-9601-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785992227050689-9601-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.96.126:38397
--webserver_interface=127.9.96.126
--webserver_port=0
--builtin_ntp_servers=127.9.96.84:39379
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.96.126:38397
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785992227050689-9601-0/testchainca.cert with env {}
W20260806 04:57:54.838222 17903 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 04:57:54.838524 17903 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 04:57:54.838614 17903 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 04:57:54.845721 17903 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 04:57:54.845815 17903 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 04:57:54.845853 17903 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 04:57:54.845921 17903 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 04:57:54.845957 17903 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260806 04:57:54.846019 17903 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260806 04:57:54.846060 17903 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260806 04:57:54.855827 17903 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.96.84:39379
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785992227050689-9601-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785992227050689-9601-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.96.126:38397
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.96.126:38397
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785992227050689-9601-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.96.126
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785992227050689-9601-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785992227050689-9601-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 03:56:12 UTC on ccycloud.quasar-guijbm.root.comops.site
build id 4569618
ASAN enabled
I20260806 04:57:54.857017 17903 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 04:57:54.858228 17903 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 04:57:54.869649 17910 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:57:54.869609 17911 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:57:54.869589 17914 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:57:54.870734 17903 server_base.cc:1034] running on GCE node
I20260806 04:57:54.871552 17903 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 04:57:54.872816 17903 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 04:57:54.874058 17903 hybrid_clock.cc:648] HybridClock initialized: now 1785992274874025 us; error 46 us; skew 500 ppm
I20260806 04:57:54.874408 17903 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 04:57:54.876576 17903 webserver.cc:460] Webserver started at http://127.9.96.126:44247/ using document root <none> and password file <none>
I20260806 04:57:54.877244 17903 fs_manager.cc:359] Metadata directory not provided
I20260806 04:57:54.877388 17903 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 04:57:54.877699 17903 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 04:57:54.880520 17903 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785992227050689-9601-0/minicluster-data/master-0/data/instance:
uuid: "c14cbc0fed514f4a883f4c42f1a05e2c"
format_stamp: "Formatted at 2026-08-06 04:57:54 on dist-test-slave-x4qh"
server_key: "cb656d26fe337f60ff29f76cbacabbcf"
server_key_iv: "5e8c9c274582ccb307baa79c374fba6f"
server_key_version: "encryptionkey@0"
I20260806 04:57:54.881317 17903 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785992227050689-9601-0/minicluster-data/master-0/wal/instance:
uuid: "c14cbc0fed514f4a883f4c42f1a05e2c"
format_stamp: "Formatted at 2026-08-06 04:57:54 on dist-test-slave-x4qh"
server_key: "cb656d26fe337f60ff29f76cbacabbcf"
server_key_iv: "5e8c9c274582ccb307baa79c374fba6f"
server_key_version: "encryptionkey@0"
I20260806 04:57:54.886236 17903 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.002s
I20260806 04:57:54.889516 17923 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 04:57:54.890779 17903 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260806 04:57:54.890972 17903 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785992227050689-9601-0/minicluster-data/master-0/data,/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785992227050689-9601-0/minicluster-data/master-0/wal
uuid: "c14cbc0fed514f4a883f4c42f1a05e2c"
format_stamp: "Formatted at 2026-08-06 04:57:54 on dist-test-slave-x4qh"
server_key: "cb656d26fe337f60ff29f76cbacabbcf"
server_key_iv: "5e8c9c274582ccb307baa79c374fba6f"
server_key_version: "encryptionkey@0"
I20260806 04:57:54.891175 17903 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785992227050689-9601-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785992227050689-9601-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785992227050689-9601-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 04:57:54.908833 17903 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 04:57:54.909659 17903 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 04:57:54.910089 17903 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 04:57:54.928707 17903 rpc_server.cc:307] RPC server started. Bound to: 127.9.96.126:38397
I20260806 04:57:54.928740 18001 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.96.126:38397 every 8 connection(s)
I20260806 04:57:54.930023 17903 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785992227050689-9601-0/minicluster-data/master-0/data/info.pb
I20260806 04:57:54.933495 18004 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 04:57:54.935242 9601 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu as pid 17903
I20260806 04:57:54.935489 9601 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785992227050689-9601-0/minicluster-data/master-0/wal/instance
I20260806 04:57:54.936151 9601 external_mini_cluster.cc:1428] Setting key e14f470cd419554ad503dd4690e091e5
I20260806 04:57:54.946578 18004 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c14cbc0fed514f4a883f4c42f1a05e2c: Bootstrap starting.
I20260806 04:57:54.950184 18004 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c14cbc0fed514f4a883f4c42f1a05e2c: Neither blocks nor log segments found. Creating new log.
I20260806 04:57:54.951542 18004 log.cc:824] T 00000000000000000000000000000000 P c14cbc0fed514f4a883f4c42f1a05e2c: Log is configured to *not* fsync() on all Append() calls
I20260806 04:57:54.951537 18007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:57:54.938092 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43230 (local address 127.9.96.126:38397)
0806 04:57:54.939163 (+ 1071us) server_negotiation.cc:206] Beginning negotiation
0806 04:57:54.939186 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0806 04:57:54.939238 (+ 52us) server_negotiation.cc:407] Connection header received
0806 04:57:54.941536 (+ 2298us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:57:54.941561 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:57:54.942157 (+ 596us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:57:54.942589 (+ 432us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:57:54.943125 (+ 536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:57:54.945196 (+ 2071us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:57:54.946424 (+ 1228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:57:54.947474 (+ 1050us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:57:54.947643 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:57:54.948748 (+ 1105us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:57:54.948757 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:57:54.948773 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 04:57:54.949456 (+ 683us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:57:54.949568 (+ 112us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 04:57:54.949592 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:57:54.949697 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0806 04:57:54.949898 (+ 201us) server_negotiation.cc:299] Negotiation successful
0806 04:57:54.950239 (+ 341us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":554,"thread_start_us":327,"threads_started":1}
I20260806 04:57:54.954852 18004 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c14cbc0fed514f4a883f4c42f1a05e2c: No bootstrap required, opened a new log
I20260806 04:57:54.955721 9601 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 04:57:54.962594 18004 raft_consensus.cc:348] T 00000000000000000000000000000000 P c14cbc0fed514f4a883f4c42f1a05e2c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c14cbc0fed514f4a883f4c42f1a05e2c" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 38397 } }
I20260806 04:57:54.962925 18004 raft_consensus.cc:374] T 00000000000000000000000000000000 P c14cbc0fed514f4a883f4c42f1a05e2c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 04:57:54.963028 18004 raft_consensus.cc:729] T 00000000000000000000000000000000 P c14cbc0fed514f4a883f4c42f1a05e2c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c14cbc0fed514f4a883f4c42f1a05e2c, State: Initialized, Role: FOLLOWER
I20260806 04:57:54.963640 18004 consensus_queue.cc:260] T 00000000000000000000000000000000 P c14cbc0fed514f4a883f4c42f1a05e2c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c14cbc0fed514f4a883f4c42f1a05e2c" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 38397 } }
I20260806 04:57:54.963873 18004 raft_consensus.cc:388] T 00000000000000000000000000000000 P c14cbc0fed514f4a883f4c42f1a05e2c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 04:57:54.964021 18004 raft_consensus.cc:482] T 00000000000000000000000000000000 P c14cbc0fed514f4a883f4c42f1a05e2c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 04:57:54.964198 18004 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c14cbc0fed514f4a883f4c42f1a05e2c [term 0 FOLLOWER]: Advancing to term 1
I20260806 04:57:54.966321 18004 raft_consensus.cc:504] T 00000000000000000000000000000000 P c14cbc0fed514f4a883f4c42f1a05e2c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c14cbc0fed514f4a883f4c42f1a05e2c" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 38397 } }
I20260806 04:57:54.966950 18004 leader_election.cc:302] T 00000000000000000000000000000000 P c14cbc0fed514f4a883f4c42f1a05e2c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c14cbc0fed514f4a883f4c42f1a05e2c; no voters:
I20260806 04:57:54.967592 18004 leader_election.cc:288] T 00000000000000000000000000000000 P c14cbc0fed514f4a883f4c42f1a05e2c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 04:57:54.967687 18010 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c14cbc0fed514f4a883f4c42f1a05e2c [term 1 FOLLOWER]: Leader election won for term 1
I20260806 04:57:54.968489 18010 raft_consensus.cc:686] T 00000000000000000000000000000000 P c14cbc0fed514f4a883f4c42f1a05e2c [term 1 LEADER]: Becoming Leader. State: Replica: c14cbc0fed514f4a883f4c42f1a05e2c, State: Running, Role: LEADER
I20260806 04:57:54.969026 18010 consensus_queue.cc:237] T 00000000000000000000000000000000 P c14cbc0fed514f4a883f4c42f1a05e2c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c14cbc0fed514f4a883f4c42f1a05e2c" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 38397 } }
I20260806 04:57:54.969683 18004 sys_catalog.cc:564] T 00000000000000000000000000000000 P c14cbc0fed514f4a883f4c42f1a05e2c [sys.catalog]: configured and running, proceeding with master startup.
I20260806 04:57:54.971863 18011 sys_catalog.cc:455] T 00000000000000000000000000000000 P c14cbc0fed514f4a883f4c42f1a05e2c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c14cbc0fed514f4a883f4c42f1a05e2c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c14cbc0fed514f4a883f4c42f1a05e2c" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 38397 } } }
I20260806 04:57:54.972223 18011 sys_catalog.cc:458] T 00000000000000000000000000000000 P c14cbc0fed514f4a883f4c42f1a05e2c [sys.catalog]: This master's current role is: LEADER
I20260806 04:57:54.971939 18012 sys_catalog.cc:455] T 00000000000000000000000000000000 P c14cbc0fed514f4a883f4c42f1a05e2c [sys.catalog]: SysCatalogTable state changed. Reason: New leader c14cbc0fed514f4a883f4c42f1a05e2c. Latest consensus state: current_term: 1 leader_uuid: "c14cbc0fed514f4a883f4c42f1a05e2c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c14cbc0fed514f4a883f4c42f1a05e2c" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 38397 } } }
I20260806 04:57:54.972355 18012 sys_catalog.cc:458] T 00000000000000000000000000000000 P c14cbc0fed514f4a883f4c42f1a05e2c [sys.catalog]: This master's current role is: LEADER
I20260806 04:57:54.976683 18018 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 04:57:54.980979 18018 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 04:57:54.988790 18018 catalog_manager.cc:1347] Generated new cluster ID: 8fe442237a8647d38417c228f610f51e
I20260806 04:57:54.988897 18018 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 04:57:55.026160 18018 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 04:57:55.028095 18018 catalog_manager.cc:1504] Loading token signing keys...
I20260806 04:57:55.037146 18018 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c14cbc0fed514f4a883f4c42f1a05e2c: Generated new TSK 0
I20260806 04:57:55.038154 18018 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 04:57:55.159956 18007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:57:55.051517 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43234 (local address 127.9.96.126:38397)
0806 04:57:55.051704 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0806 04:57:55.051710 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 04:57:55.052132 (+ 422us) server_negotiation.cc:407] Connection header received
0806 04:57:55.052321 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:57:55.052327 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:57:55.052423 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:57:55.052589 (+ 166us) server_negotiation.cc:226] Negotiated authn=JWT
0806 04:57:55.053215 (+ 626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:57:55.054356 (+ 1141us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:57:55.055790 (+ 1434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:57:55.056489 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:57:55.056603 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:57:55.057025 (+ 422us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0806 04:57:55.159120 (+102095us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0806 04:57:55.159616 (+ 496us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:43234: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":61}
W20260806 04:57:55.160120 18007 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:43234: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260806 04:57:55.160266 18049 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 04:57:55.051345 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:38397 (local address 127.0.0.1:43234)
0806 04:57:55.052037 (+ 692us) negotiation.cc:107] Waiting for socket to connect
0806 04:57:55.052052 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 04:57:55.052167 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:57:55.052649 (+ 482us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:57:55.052657 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:57:55.052669 (+ 12us) client_negotiation.cc:189] Negotiated authn=JWT
0806 04:57:55.053056 (+ 387us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:57:55.053066 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:57:55.054502 (+ 1436us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:57:55.054506 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:57:55.055640 (+ 1134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:57:55.055652 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:57:55.056643 (+ 991us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:57:55.056647 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:57:55.056771 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:57:55.056798 (+ 27us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0806 04:57:55.159471 (+102673us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0806 04:57:55.159769 (+ 298us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.9.96.126:38397: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":557,"thread_start_us":393,"threads_started":1}
I20260806 04:57:55.161676 9601 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu pid 17903
I20260806 04:57:55.162892 17997 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 04:57:55.263895 9601 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu with pid 17903
2026-08-06T04:57:55Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (823 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (410 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (423 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785992227050689-9601-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 04:57:56 dist-test-slave-x4qh krb5kdc[18080](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 04:57:56 dist-test-slave-x4qh krb5kdc[18080](info): set up 2 sockets
Aug 06 04:57:56 dist-test-slave-x4qh krb5kdc[18080](info): commencing operation
krb5kdc: starting...
W20260806 04:57:57.374343 9601 mini_kdc.cc:121] Time spent starting KDC: real 1.233s user 0.000s sys 0.024s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785992227050689-9601-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785992227050689-9601-0/krb5kdc/test-user.keytab.
Aug 06 04:57:57 dist-test-slave-x4qh krb5kdc[18080](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992277, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T04:57:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T04:57:57Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.96.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.96.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.96.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.126.keytab.
Entry for principal kudu/127.9.96.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.126.keytab.
I20260806 04:57:57.592514 9601 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
/tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785992227050689-9601-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785992227050689-9601-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785992227050689-9601-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785992227050689-9601-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.96.126:37487
--webserver_interface=127.9.96.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.126.keytab
--principal=kudu/127.9.96.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.96.84:44267
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.96.126:37487
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785992227050689-9601-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785992227050689-9601-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785992227050689-9601-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 04:57:57.908417 18098 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 04:57:57.908769 18098 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 04:57:57.908937 18098 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 04:57:57.916152 18098 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 04:57:57.916285 18098 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 04:57:57.916399 18098 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 04:57:57.916517 18098 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 04:57:57.916607 18098 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 04:57:57.926234 18098 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.96.84:44267
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785992227050689-9601-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785992227050689-9601-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.96.126:37487
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.126.keytab
--principal=kudu/127.9.96.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.96.126:37487
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785992227050689-9601-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.96.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785992227050689-9601-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 03:56:12 UTC on ccycloud.quasar-guijbm.root.comops.site
build id 4569618
ASAN enabled
I20260806 04:57:57.927531 18098 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 04:57:57.928866 18098 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 04:57:57.940918 18106 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:57:57.941128 18105 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:57:57.940913 18108 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:57:57.941619 18098 server_base.cc:1034] running on GCE node
I20260806 04:57:57.942442 18098 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 04:57:57.943826 18098 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 04:57:57.945032 18098 hybrid_clock.cc:648] HybridClock initialized: now 1785992277944992 us; error 37 us; skew 500 ppm
I20260806 04:57:57.945365 18098 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 04:57:57 dist-test-slave-x4qh krb5kdc[18080](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992277, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.96.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 04:57:57.948520 18098 init.cc:377] Logged in from keytab as kudu/127.9.96.126@KRBTEST.COM (short username kudu)
I20260806 04:57:57.950001 18098 webserver.cc:460] Webserver started at http://127.9.96.126:42617/ using document root <none> and password file <none>
I20260806 04:57:57.950660 18098 fs_manager.cc:359] Metadata directory not provided
I20260806 04:57:57.950788 18098 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 04:57:57.951103 18098 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 04:57:57.953945 18098 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785992227050689-9601-0/minicluster-data/master-0/data/instance:
uuid: "a33158b7ef9b43eca2d51ac049270ef5"
format_stamp: "Formatted at 2026-08-06 04:57:57 on dist-test-slave-x4qh"
server_key: "cc46d73d2eb48958aca208483fb64c4e"
server_key_iv: "1a4b3adc20b0bab9fb37b0e941b6cce2"
server_key_version: "encryptionkey@0"
I20260806 04:57:57.954696 18098 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785992227050689-9601-0/minicluster-data/master-0/wal/instance:
uuid: "a33158b7ef9b43eca2d51ac049270ef5"
format_stamp: "Formatted at 2026-08-06 04:57:57 on dist-test-slave-x4qh"
server_key: "cc46d73d2eb48958aca208483fb64c4e"
server_key_iv: "1a4b3adc20b0bab9fb37b0e941b6cce2"
server_key_version: "encryptionkey@0"
I20260806 04:57:57.959730 18098 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260806 04:57:57.962867 18120 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 04:57:57.964079 18098 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 04:57:57.964255 18098 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785992227050689-9601-0/minicluster-data/master-0/data,/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785992227050689-9601-0/minicluster-data/master-0/wal
uuid: "a33158b7ef9b43eca2d51ac049270ef5"
format_stamp: "Formatted at 2026-08-06 04:57:57 on dist-test-slave-x4qh"
server_key: "cc46d73d2eb48958aca208483fb64c4e"
server_key_iv: "1a4b3adc20b0bab9fb37b0e941b6cce2"
server_key_version: "encryptionkey@0"
I20260806 04:57:57.964480 18098 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785992227050689-9601-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785992227050689-9601-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785992227050689-9601-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 04:57:57.981479 18098 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 04:57:57.985796 18098 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 04:57:57.986097 18098 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 04:57:58.004659 18098 rpc_server.cc:307] RPC server started. Bound to: 127.9.96.126:37487
I20260806 04:57:58.004699 18201 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.96.126:37487 every 8 connection(s)
I20260806 04:57:58.006039 18098 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785992227050689-9601-0/minicluster-data/master-0/data/info.pb
I20260806 04:57:58.009706 18202 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 04:57:58.011089 9601 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu as pid 18098
I20260806 04:57:58.011369 9601 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785992227050689-9601-0/minicluster-data/master-0/wal/instance
I20260806 04:57:58.012243 9601 external_mini_cluster.cc:1428] Setting key e66cfd17049ea37286882262159c6664
I20260806 04:57:58.022779 18202 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a33158b7ef9b43eca2d51ac049270ef5: Bootstrap starting.
Aug 06 04:57:58 dist-test-slave-x4qh krb5kdc[18080](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992277, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.96.126@KRBTEST.COM
I20260806 04:57:58.025892 18202 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a33158b7ef9b43eca2d51ac049270ef5: Neither blocks nor log segments found. Creating new log.
I20260806 04:57:58.027164 18202 log.cc:824] T 00000000000000000000000000000000 P a33158b7ef9b43eca2d51ac049270ef5: Log is configured to *not* fsync() on all Append() calls
I20260806 04:57:58.030077 18202 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a33158b7ef9b43eca2d51ac049270ef5: No bootstrap required, opened a new log
I20260806 04:57:58.034353 18207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:57:58.014399 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35962 (local address 127.9.96.126:37487)
0806 04:57:58.015176 (+ 777us) server_negotiation.cc:206] Beginning negotiation
0806 04:57:58.015188 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 04:57:58.015233 (+ 45us) server_negotiation.cc:407] Connection header received
0806 04:57:58.016927 (+ 1694us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:57:58.016949 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:57:58.017317 (+ 368us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:57:58.017700 (+ 383us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:57:58.019150 (+ 1450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:57:58.020553 (+ 1403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:57:58.021799 (+ 1246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:57:58.022465 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:57:58.022658 (+ 193us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:57:58.027036 (+ 4378us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:57:58.027044 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:57:58.027058 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:57:58.027091 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:57:58.030469 (+ 3378us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:57:58.031240 (+ 771us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:57:58.031246 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:57:58.031251 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:57:58.031486 (+ 235us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:57:58.031889 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:57:58.031892 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:57:58.031894 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:57:58.032428 (+ 534us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:57:58.032660 (+ 232us) server_negotiation.cc:1035] Waiting for connection context
0806 04:57:58.033098 (+ 438us) server_negotiation.cc:299] Negotiation successful
0806 04:57:58.033409 (+ 311us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":571,"thread_start_us":345,"threads_started":1}
I20260806 04:57:58.038247 18202 raft_consensus.cc:348] T 00000000000000000000000000000000 P a33158b7ef9b43eca2d51ac049270ef5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a33158b7ef9b43eca2d51ac049270ef5" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 37487 } }
I20260806 04:57:58.038589 18202 raft_consensus.cc:374] T 00000000000000000000000000000000 P a33158b7ef9b43eca2d51ac049270ef5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 04:57:58.038694 18202 raft_consensus.cc:729] T 00000000000000000000000000000000 P a33158b7ef9b43eca2d51ac049270ef5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a33158b7ef9b43eca2d51ac049270ef5, State: Initialized, Role: FOLLOWER
I20260806 04:57:58.039211 18202 consensus_queue.cc:260] T 00000000000000000000000000000000 P a33158b7ef9b43eca2d51ac049270ef5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a33158b7ef9b43eca2d51ac049270ef5" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 37487 } }
I20260806 04:57:58.039479 18202 raft_consensus.cc:388] T 00000000000000000000000000000000 P a33158b7ef9b43eca2d51ac049270ef5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 04:57:58.039628 18202 raft_consensus.cc:482] T 00000000000000000000000000000000 P a33158b7ef9b43eca2d51ac049270ef5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 04:57:58.039808 18202 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a33158b7ef9b43eca2d51ac049270ef5 [term 0 FOLLOWER]: Advancing to term 1
I20260806 04:57:58.041875 18202 raft_consensus.cc:504] T 00000000000000000000000000000000 P a33158b7ef9b43eca2d51ac049270ef5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a33158b7ef9b43eca2d51ac049270ef5" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 37487 } }
I20260806 04:57:58.042354 18202 leader_election.cc:302] T 00000000000000000000000000000000 P a33158b7ef9b43eca2d51ac049270ef5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a33158b7ef9b43eca2d51ac049270ef5; no voters:
I20260806 04:57:58.043007 18202 leader_election.cc:288] T 00000000000000000000000000000000 P a33158b7ef9b43eca2d51ac049270ef5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 04:57:58.043155 18209 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a33158b7ef9b43eca2d51ac049270ef5 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 04:57:58.043915 18209 raft_consensus.cc:686] T 00000000000000000000000000000000 P a33158b7ef9b43eca2d51ac049270ef5 [term 1 LEADER]: Becoming Leader. State: Replica: a33158b7ef9b43eca2d51ac049270ef5, State: Running, Role: LEADER
I20260806 04:57:58.044445 18209 consensus_queue.cc:237] T 00000000000000000000000000000000 P a33158b7ef9b43eca2d51ac049270ef5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a33158b7ef9b43eca2d51ac049270ef5" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 37487 } }
I20260806 04:57:58.045079 18202 sys_catalog.cc:564] T 00000000000000000000000000000000 P a33158b7ef9b43eca2d51ac049270ef5 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 04:57:58.047086 18210 sys_catalog.cc:455] T 00000000000000000000000000000000 P a33158b7ef9b43eca2d51ac049270ef5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a33158b7ef9b43eca2d51ac049270ef5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a33158b7ef9b43eca2d51ac049270ef5" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 37487 } } }
I20260806 04:57:58.047575 18210 sys_catalog.cc:458] T 00000000000000000000000000000000 P a33158b7ef9b43eca2d51ac049270ef5 [sys.catalog]: This master's current role is: LEADER
I20260806 04:57:58.048060 18211 sys_catalog.cc:455] T 00000000000000000000000000000000 P a33158b7ef9b43eca2d51ac049270ef5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a33158b7ef9b43eca2d51ac049270ef5. Latest consensus state: current_term: 1 leader_uuid: "a33158b7ef9b43eca2d51ac049270ef5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a33158b7ef9b43eca2d51ac049270ef5" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 37487 } } }
I20260806 04:57:58.048360 18211 sys_catalog.cc:458] T 00000000000000000000000000000000 P a33158b7ef9b43eca2d51ac049270ef5 [sys.catalog]: This master's current role is: LEADER
I20260806 04:57:58.049715 18221 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 04:57:58.054896 18221 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 04:57:58.063037 18221 catalog_manager.cc:1347] Generated new cluster ID: 4b3a3f7443c14c70aa0bd50e927a58c1
I20260806 04:57:58.063161 18221 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 04:57:58.077725 18221 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 04:57:58.079133 18221 catalog_manager.cc:1504] Loading token signing keys...
I20260806 04:57:58.085789 18221 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a33158b7ef9b43eca2d51ac049270ef5: Generated new TSK 0
I20260806 04:57:58.086453 18221 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.9.96.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.96.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.96.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.65.keytab.
Entry for principal kudu/127.9.96.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.65.keytab.
I20260806 04:57:58.138851 9601 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
/tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785992227050689-9601-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785992227050689-9601-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785992227050689-9601-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785992227050689-9601-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.96.65:0
--local_ip_for_outbound_sockets=127.9.96.65
--webserver_interface=127.9.96.65
--webserver_port=0
--tserver_master_addrs=127.9.96.126:37487
--keytab_file=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.65.keytab
--principal=kudu/127.9.96.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.96.84:44267
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785992227050689-9601-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785992227050689-9601-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785992227050689-9601-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 04:57:58.461350 18239 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 04:57:58.461592 18239 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 04:57:58.461709 18239 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 04:57:58.469017 18239 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 04:57:58.469116 18239 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 04:57:58.469237 18239 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.96.65
I20260806 04:57:58.479837 18239 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.96.84:44267
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785992227050689-9601-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785992227050689-9601-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.65.keytab
--principal=kudu/127.9.96.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.96.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785992227050689-9601-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.96.65
--webserver_port=0
--tserver_master_addrs=127.9.96.126:37487
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.96.65
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785992227050689-9601-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 03:56:12 UTC on ccycloud.quasar-guijbm.root.comops.site
build id 4569618
ASAN enabled
I20260806 04:57:58.480993 18239 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 04:57:58.482174 18239 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 04:57:58.495749 18247 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:57:58.495726 18248 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:57:58.495692 18252 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:57:58.496569 18239 server_base.cc:1034] running on GCE node
I20260806 04:57:58.497202 18239 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 04:57:58.498118 18239 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 04:57:58.499342 18239 hybrid_clock.cc:648] HybridClock initialized: now 1785992278499290 us; error 62 us; skew 500 ppm
I20260806 04:57:58.499699 18239 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 04:57:58 dist-test-slave-x4qh krb5kdc[18080](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992278, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.96.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 04:57:58.502717 18239 init.cc:377] Logged in from keytab as kudu/127.9.96.65@KRBTEST.COM (short username kudu)
I20260806 04:57:58.504136 18239 webserver.cc:460] Webserver started at http://127.9.96.65:44603/ using document root <none> and password file <none>
I20260806 04:57:58.504755 18239 fs_manager.cc:359] Metadata directory not provided
I20260806 04:57:58.504858 18239 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 04:57:58.505103 18239 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 04:57:58.508044 18239 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785992227050689-9601-0/minicluster-data/ts-0/data/instance:
uuid: "7a14c869085845728d32fbde8358fd5f"
format_stamp: "Formatted at 2026-08-06 04:57:58 on dist-test-slave-x4qh"
server_key: "13cc5f0935e7c882a03d13a54168e913"
server_key_iv: "e0bf252ba162618e6979b6ae2362af2a"
server_key_version: "encryptionkey@0"
I20260806 04:57:58.508850 18239 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785992227050689-9601-0/minicluster-data/ts-0/wal/instance:
uuid: "7a14c869085845728d32fbde8358fd5f"
format_stamp: "Formatted at 2026-08-06 04:57:58 on dist-test-slave-x4qh"
server_key: "13cc5f0935e7c882a03d13a54168e913"
server_key_iv: "e0bf252ba162618e6979b6ae2362af2a"
server_key_version: "encryptionkey@0"
I20260806 04:57:58.513568 18239 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.004s
I20260806 04:57:58.516655 18263 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 04:57:58.517913 18239 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260806 04:57:58.518041 18239 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785992227050689-9601-0/minicluster-data/ts-0/data,/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785992227050689-9601-0/minicluster-data/ts-0/wal
uuid: "7a14c869085845728d32fbde8358fd5f"
format_stamp: "Formatted at 2026-08-06 04:57:58 on dist-test-slave-x4qh"
server_key: "13cc5f0935e7c882a03d13a54168e913"
server_key_iv: "e0bf252ba162618e6979b6ae2362af2a"
server_key_version: "encryptionkey@0"
I20260806 04:57:58.518198 18239 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785992227050689-9601-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785992227050689-9601-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785992227050689-9601-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 04:57:58.551347 18239 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 04:57:58.555766 18239 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 04:57:58.556030 18239 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 04:57:58.556923 18239 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 04:57:58.558139 18239 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 04:57:58.558219 18239 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 04:57:58.558318 18239 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 04:57:58.558364 18239 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 04:57:58.591719 18239 rpc_server.cc:307] RPC server started. Bound to: 127.9.96.65:40763
I20260806 04:57:58.591763 18441 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.96.65:40763 every 8 connection(s)
I20260806 04:57:58.592988 18239 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785992227050689-9601-0/minicluster-data/ts-0/data/info.pb
I20260806 04:57:58.600410 9601 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu as pid 18239
I20260806 04:57:58.600654 9601 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785992227050689-9601-0/minicluster-data/ts-0/wal/instance
I20260806 04:57:58.601339 9601 external_mini_cluster.cc:1428] Setting key 39e675231fcde2a88a17398f6b42c339
Aug 06 04:57:58 dist-test-slave-x4qh krb5kdc[18080](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992278, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.96.65@KRBTEST.COM for kudu/127.9.96.126@KRBTEST.COM
I20260806 04:57:58.614161 18444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:57:58.595473 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.96.65:54619 (local address 127.9.96.126:37487)
0806 04:57:58.595951 (+ 478us) server_negotiation.cc:206] Beginning negotiation
0806 04:57:58.595958 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 04:57:58.596874 (+ 916us) server_negotiation.cc:407] Connection header received
0806 04:57:58.598585 (+ 1711us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:57:58.598590 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:57:58.598667 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:57:58.598769 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:57:58.600437 (+ 1668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:57:58.601236 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:57:58.603151 (+ 1915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:57:58.603723 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:57:58.603881 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:57:58.607226 (+ 3345us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:57:58.607235 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:57:58.607242 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:57:58.607287 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:57:58.610878 (+ 3591us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:57:58.611645 (+ 767us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:57:58.611653 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:57:58.611660 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:57:58.611813 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:57:58.612389 (+ 576us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:57:58.612393 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:57:58.612395 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:57:58.612647 (+ 252us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:57:58.612766 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0806 04:57:58.613693 (+ 927us) server_negotiation.cc:299] Negotiation successful
0806 04:57:58.613922 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":351,"thread_start_us":241,"threads_started":1}
I20260806 04:57:58.615739 18448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:57:58.596049 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:37487 (local address 127.9.96.65:54619)
0806 04:57:58.596724 (+ 675us) negotiation.cc:107] Waiting for socket to connect
0806 04:57:58.596761 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0806 04:57:58.598331 (+ 1570us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:57:58.598974 (+ 643us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:57:58.598981 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:57:58.599557 (+ 576us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:57:58.600249 (+ 692us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:57:58.600277 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:57:58.601395 (+ 1118us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:57:58.601401 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:57:58.603009 (+ 1608us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:57:58.603024 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:57:58.603942 (+ 918us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:57:58.603948 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:57:58.604140 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:57:58.604834 (+ 694us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:57:58.604862 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:57:58.606997 (+ 2135us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 04:57:58.611030 (+ 4033us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:57:58.611037 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:57:58.611049 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:57:58.611531 (+ 482us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:57:58.611960 (+ 429us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:57:58.611963 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:57:58.611966 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:57:58.612258 (+ 292us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:57:58.612774 (+ 516us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 04:57:58.612783 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 04:57:58.613285 (+ 502us) client_negotiation.cc:769] Sending connection context
0806 04:57:58.613611 (+ 326us) client_negotiation.cc:240] Negotiation successful
0806 04:57:58.614231 (+ 620us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":464,"thread_start_us":304,"threads_started":1}
I20260806 04:57:58.617303 18442 heartbeater.cc:347] Connected to a master server at 127.9.96.126:37487
I20260806 04:57:58.617605 18442 heartbeater.cc:464] Registering TS with master...
I20260806 04:57:58.618369 18442 heartbeater.cc:511] Master 127.9.96.126:37487 requested a full tablet report, sending...
I20260806 04:57:58.620434 18146 ts_manager.cc:194] Registered new tserver with Master: 7a14c869085845728d32fbde8358fd5f (127.9.96.65:40763)
I20260806 04:57:58.622032 18146 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.96.65@KRBTEST.COM'} at 127.9.96.65:54619
WARNING: no policy specified for kudu/127.9.96.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.96.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.96.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.66.keytab.
Entry for principal kudu/127.9.96.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.66.keytab.
I20260806 04:57:58.654579 9601 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
/tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785992227050689-9601-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785992227050689-9601-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785992227050689-9601-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785992227050689-9601-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.96.66:0
--local_ip_for_outbound_sockets=127.9.96.66
--webserver_interface=127.9.96.66
--webserver_port=0
--tserver_master_addrs=127.9.96.126:37487
--keytab_file=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.66.keytab
--principal=kudu/127.9.96.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.96.84:44267
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785992227050689-9601-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785992227050689-9601-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785992227050689-9601-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 04:57:58.979015 18452 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 04:57:58.979328 18452 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 04:57:58.979511 18452 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 04:57:58.986517 18452 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 04:57:58.986657 18452 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 04:57:58.986862 18452 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.96.66
I20260806 04:57:58.996285 18452 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.96.84:44267
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785992227050689-9601-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785992227050689-9601-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.66.keytab
--principal=kudu/127.9.96.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.96.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785992227050689-9601-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.96.66
--webserver_port=0
--tserver_master_addrs=127.9.96.126:37487
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.96.66
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785992227050689-9601-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 03:56:12 UTC on ccycloud.quasar-guijbm.root.comops.site
build id 4569618
ASAN enabled
I20260806 04:57:58.997474 18452 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 04:57:58.998795 18452 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 04:57:59.012578 18466 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:57:59.012569 18464 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:57:59.015682 18463 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:57:59.020993 18452 server_base.cc:1034] running on GCE node
I20260806 04:57:59.021680 18452 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 04:57:59.022737 18452 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 04:57:59.023993 18452 hybrid_clock.cc:648] HybridClock initialized: now 1785992279023930 us; error 51 us; skew 500 ppm
I20260806 04:57:59.024358 18452 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 04:57:59 dist-test-slave-x4qh krb5kdc[18080](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992279, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.96.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 04:57:59.027671 18452 init.cc:377] Logged in from keytab as kudu/127.9.96.66@KRBTEST.COM (short username kudu)
I20260806 04:57:59.029241 18452 webserver.cc:460] Webserver started at http://127.9.96.66:35433/ using document root <none> and password file <none>
I20260806 04:57:59.030027 18452 fs_manager.cc:359] Metadata directory not provided
I20260806 04:57:59.030247 18452 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 04:57:59.030586 18452 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 04:57:59.033564 18452 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785992227050689-9601-0/minicluster-data/ts-1/data/instance:
uuid: "d8e455fbdb5f4ab6ae7f8aa8b199a924"
format_stamp: "Formatted at 2026-08-06 04:57:59 on dist-test-slave-x4qh"
server_key: "07b946b9d069f92cfafa0877c94a0453"
server_key_iv: "74e17c9f70df7bf8b9fb68eb8abe1f70"
server_key_version: "encryptionkey@0"
I20260806 04:57:59.034346 18452 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785992227050689-9601-0/minicluster-data/ts-1/wal/instance:
uuid: "d8e455fbdb5f4ab6ae7f8aa8b199a924"
format_stamp: "Formatted at 2026-08-06 04:57:59 on dist-test-slave-x4qh"
server_key: "07b946b9d069f92cfafa0877c94a0453"
server_key_iv: "74e17c9f70df7bf8b9fb68eb8abe1f70"
server_key_version: "encryptionkey@0"
I20260806 04:57:59.039245 18452 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260806 04:57:59.042496 18475 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 04:57:59.043802 18452 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 04:57:59.043962 18452 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785992227050689-9601-0/minicluster-data/ts-1/data,/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785992227050689-9601-0/minicluster-data/ts-1/wal
uuid: "d8e455fbdb5f4ab6ae7f8aa8b199a924"
format_stamp: "Formatted at 2026-08-06 04:57:59 on dist-test-slave-x4qh"
server_key: "07b946b9d069f92cfafa0877c94a0453"
server_key_iv: "74e17c9f70df7bf8b9fb68eb8abe1f70"
server_key_version: "encryptionkey@0"
I20260806 04:57:59.044169 18452 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785992227050689-9601-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785992227050689-9601-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785992227050689-9601-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 04:57:59.058007 18452 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 04:57:59.062110 18452 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 04:57:59.062394 18452 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 04:57:59.063318 18452 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 04:57:59.064600 18452 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 04:57:59.064704 18452 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 04:57:59.064843 18452 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 04:57:59.064916 18452 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 04:57:59.096856 18452 rpc_server.cc:307] RPC server started. Bound to: 127.9.96.66:37275
I20260806 04:57:59.096936 18646 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.96.66:37275 every 8 connection(s)
I20260806 04:57:59.098107 18452 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785992227050689-9601-0/minicluster-data/ts-1/data/info.pb
I20260806 04:57:59.106884 9601 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu as pid 18452
I20260806 04:57:59.107106 9601 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785992227050689-9601-0/minicluster-data/ts-1/wal/instance
I20260806 04:57:59.107630 9601 external_mini_cluster.cc:1428] Setting key 2d936c93fa43d306d0d0225de3602e79
Aug 06 04:57:59 dist-test-slave-x4qh krb5kdc[18080](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992279, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.96.66@KRBTEST.COM for kudu/127.9.96.126@KRBTEST.COM
I20260806 04:57:59.117246 18444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:57:59.100618 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.96.66:36237 (local address 127.9.96.126:37487)
0806 04:57:59.100837 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0806 04:57:59.100844 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 04:57:59.101997 (+ 1153us) server_negotiation.cc:407] Connection header received
0806 04:57:59.103790 (+ 1793us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:57:59.103795 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:57:59.103863 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:57:59.103944 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:57:59.105313 (+ 1369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:57:59.106007 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:57:59.107676 (+ 1669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:57:59.108179 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:57:59.108327 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:57:59.111644 (+ 3317us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:57:59.111649 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:57:59.111653 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:57:59.111683 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:57:59.114284 (+ 2601us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:57:59.115010 (+ 726us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:57:59.115014 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:57:59.115017 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:57:59.115145 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:57:59.115640 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:57:59.115644 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:57:59.115646 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:57:59.115908 (+ 262us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:57:59.116058 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0806 04:57:59.116692 (+ 634us) server_negotiation.cc:299] Negotiation successful
0806 04:57:59.116916 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260806 04:57:59.118237 18650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:57:59.101153 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:37487 (local address 127.9.96.66:36237)
0806 04:57:59.101845 (+ 692us) negotiation.cc:107] Waiting for socket to connect
0806 04:57:59.101886 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0806 04:57:59.103536 (+ 1650us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:57:59.104134 (+ 598us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:57:59.104141 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:57:59.104628 (+ 487us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:57:59.105159 (+ 531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:57:59.105175 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:57:59.106140 (+ 965us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:57:59.106144 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:57:59.107522 (+ 1378us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:57:59.107534 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:57:59.108402 (+ 868us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:57:59.108409 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:57:59.108648 (+ 239us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:57:59.109317 (+ 669us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:57:59.109339 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:57:59.111434 (+ 2095us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 04:57:59.114429 (+ 2995us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:57:59.114438 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:57:59.114452 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:57:59.114879 (+ 427us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:57:59.115256 (+ 377us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:57:59.115261 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:57:59.115265 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:57:59.115517 (+ 252us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:57:59.116034 (+ 517us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 04:57:59.116041 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 04:57:59.116349 (+ 308us) client_negotiation.cc:769] Sending connection context
0806 04:57:59.116614 (+ 265us) client_negotiation.cc:240] Negotiation successful
0806 04:57:59.117103 (+ 489us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":469,"thread_start_us":287,"threads_started":1}
I20260806 04:57:59.119726 18648 heartbeater.cc:347] Connected to a master server at 127.9.96.126:37487
I20260806 04:57:59.120062 18648 heartbeater.cc:464] Registering TS with master...
I20260806 04:57:59.120940 18648 heartbeater.cc:511] Master 127.9.96.126:37487 requested a full tablet report, sending...
I20260806 04:57:59.122622 18146 ts_manager.cc:194] Registered new tserver with Master: d8e455fbdb5f4ab6ae7f8aa8b199a924 (127.9.96.66:37275)
I20260806 04:57:59.123960 18146 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.96.66@KRBTEST.COM'} at 127.9.96.66:36237
WARNING: no policy specified for kudu/127.9.96.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.96.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.96.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.67.keytab.
Entry for principal kudu/127.9.96.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.67.keytab.
I20260806 04:57:59.160821 9601 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
/tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785992227050689-9601-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785992227050689-9601-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785992227050689-9601-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785992227050689-9601-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.96.67:0
--local_ip_for_outbound_sockets=127.9.96.67
--webserver_interface=127.9.96.67
--webserver_port=0
--tserver_master_addrs=127.9.96.126:37487
--keytab_file=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.67.keytab
--principal=kudu/127.9.96.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.96.84:44267
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785992227050689-9601-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785992227050689-9601-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785992227050689-9601-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 04:57:59.482149 18653 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 04:57:59.482476 18653 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 04:57:59.482582 18653 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 04:57:59.489773 18653 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 04:57:59.489871 18653 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 04:57:59.489989 18653 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.96.67
I20260806 04:57:59.499528 18653 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.96.84:44267
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785992227050689-9601-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785992227050689-9601-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.67.keytab
--principal=kudu/127.9.96.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.96.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785992227050689-9601-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.96.67
--webserver_port=0
--tserver_master_addrs=127.9.96.126:37487
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.96.67
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785992227050689-9601-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 03:56:12 UTC on ccycloud.quasar-guijbm.root.comops.site
build id 4569618
ASAN enabled
I20260806 04:57:59.500669 18653 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 04:57:59.501884 18653 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 04:57:59.516809 18659 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:57:59.516809 18658 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:57:59.516844 18662 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:57:59.518366 18653 server_base.cc:1034] running on GCE node
I20260806 04:57:59.518945 18653 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 04:57:59.519941 18653 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 04:57:59.521184 18653 hybrid_clock.cc:648] HybridClock initialized: now 1785992279521142 us; error 48 us; skew 500 ppm
I20260806 04:57:59.521469 18653 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 04:57:59 dist-test-slave-x4qh krb5kdc[18080](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992279, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.96.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 04:57:59.529062 18653 init.cc:377] Logged in from keytab as kudu/127.9.96.67@KRBTEST.COM (short username kudu)
I20260806 04:57:59.530438 18653 webserver.cc:460] Webserver started at http://127.9.96.67:36317/ using document root <none> and password file <none>
I20260806 04:57:59.531103 18653 fs_manager.cc:359] Metadata directory not provided
I20260806 04:57:59.531239 18653 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 04:57:59.531587 18653 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 04:57:59.534453 18653 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785992227050689-9601-0/minicluster-data/ts-2/data/instance:
uuid: "d38685eef9504815a1508bc341151271"
format_stamp: "Formatted at 2026-08-06 04:57:59 on dist-test-slave-x4qh"
server_key: "012c4f7370c183ca7fd5a7c7edfc8f98"
server_key_iv: "bbfa4eb8c6c89b7a647eb622c621ca92"
server_key_version: "encryptionkey@0"
I20260806 04:57:59.535198 18653 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785992227050689-9601-0/minicluster-data/ts-2/wal/instance:
uuid: "d38685eef9504815a1508bc341151271"
format_stamp: "Formatted at 2026-08-06 04:57:59 on dist-test-slave-x4qh"
server_key: "012c4f7370c183ca7fd5a7c7edfc8f98"
server_key_iv: "bbfa4eb8c6c89b7a647eb622c621ca92"
server_key_version: "encryptionkey@0"
I20260806 04:57:59.540295 18653 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260806 04:57:59.543569 18677 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 04:57:59.544780 18653 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 04:57:59.544946 18653 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785992227050689-9601-0/minicluster-data/ts-2/data,/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785992227050689-9601-0/minicluster-data/ts-2/wal
uuid: "d38685eef9504815a1508bc341151271"
format_stamp: "Formatted at 2026-08-06 04:57:59 on dist-test-slave-x4qh"
server_key: "012c4f7370c183ca7fd5a7c7edfc8f98"
server_key_iv: "bbfa4eb8c6c89b7a647eb622c621ca92"
server_key_version: "encryptionkey@0"
I20260806 04:57:59.545121 18653 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785992227050689-9601-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785992227050689-9601-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785992227050689-9601-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 04:57:59.558434 18653 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 04:57:59.561981 18653 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 04:57:59.562381 18653 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 04:57:59.563230 18653 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 04:57:59.564512 18653 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 04:57:59.564612 18653 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 04:57:59.564750 18653 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 04:57:59.564826 18653 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 04:57:59.596645 18653 rpc_server.cc:307] RPC server started. Bound to: 127.9.96.67:41109
I20260806 04:57:59.596681 18851 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.96.67:41109 every 8 connection(s)
I20260806 04:57:59.597808 18653 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785992227050689-9601-0/minicluster-data/ts-2/data/info.pb
I20260806 04:57:59.601141 9601 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu as pid 18653
I20260806 04:57:59.601379 9601 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785992227050689-9601-0/minicluster-data/ts-2/wal/instance
I20260806 04:57:59.601936 9601 external_mini_cluster.cc:1428] Setting key 2b0665595aeba9e055ff8dedc7d6a5b2
Aug 06 04:57:59 dist-test-slave-x4qh krb5kdc[18080](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992279, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.96.67@KRBTEST.COM for kudu/127.9.96.126@KRBTEST.COM
I20260806 04:57:59.617283 18444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:57:59.600332 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.96.67:49937 (local address 127.9.96.126:37487)
0806 04:57:59.600548 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0806 04:57:59.600555 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 04:57:59.601776 (+ 1221us) server_negotiation.cc:407] Connection header received
0806 04:57:59.603997 (+ 2221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:57:59.604003 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:57:59.604121 (+ 118us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:57:59.604228 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:57:59.605770 (+ 1542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:57:59.606534 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:57:59.608126 (+ 1592us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:57:59.608556 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:57:59.608704 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:57:59.611910 (+ 3206us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:57:59.611914 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:57:59.611918 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:57:59.611947 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:57:59.614167 (+ 2220us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:57:59.614841 (+ 674us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:57:59.614850 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:57:59.614853 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:57:59.614977 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:57:59.615472 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:57:59.615477 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:57:59.615480 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:57:59.615857 (+ 377us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:57:59.616005 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0806 04:57:59.616726 (+ 721us) server_negotiation.cc:299] Negotiation successful
0806 04:57:59.616950 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260806 04:57:59.618100 18857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:57:59.600737 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:37487 (local address 127.9.96.67:49937)
0806 04:57:59.601627 (+ 890us) negotiation.cc:107] Waiting for socket to connect
0806 04:57:59.601669 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0806 04:57:59.603710 (+ 2041us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:57:59.604465 (+ 755us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:57:59.604472 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:57:59.605082 (+ 610us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:57:59.605609 (+ 527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:57:59.605625 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:57:59.606685 (+ 1060us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:57:59.606695 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:57:59.607984 (+ 1289us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:57:59.607996 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:57:59.608798 (+ 802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:57:59.608807 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:57:59.609032 (+ 225us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:57:59.609623 (+ 591us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:57:59.609645 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:57:59.611710 (+ 2065us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 04:57:59.614315 (+ 2605us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:57:59.614323 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:57:59.614337 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:57:59.614709 (+ 372us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:57:59.615086 (+ 377us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:57:59.615091 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:57:59.615094 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:57:59.615308 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:57:59.615989 (+ 681us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 04:57:59.615996 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 04:57:59.616323 (+ 327us) client_negotiation.cc:769] Sending connection context
0806 04:57:59.616590 (+ 267us) client_negotiation.cc:240] Negotiation successful
0806 04:57:59.617090 (+ 500us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":627,"thread_start_us":355,"threads_started":1}
I20260806 04:57:59.619416 18854 heartbeater.cc:347] Connected to a master server at 127.9.96.126:37487
I20260806 04:57:59.619731 18854 heartbeater.cc:464] Registering TS with master...
I20260806 04:57:59.620404 18854 heartbeater.cc:511] Master 127.9.96.126:37487 requested a full tablet report, sending...
I20260806 04:57:59.621773 18146 ts_manager.cc:194] Registered new tserver with Master: d38685eef9504815a1508bc341151271 (127.9.96.67:41109)
I20260806 04:57:59.622557 18146 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.96.67@KRBTEST.COM'} at 127.9.96.67:49937
I20260806 04:57:59.624877 18442 heartbeater.cc:503] Master 127.9.96.126:37487 was elected leader, sending a full tablet report...
I20260806 04:57:59.628883 9601 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785992227050689-9601-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785992227050689-9601-0/krb5kdc/test-admin.keytab.
Aug 06 04:57:59 dist-test-slave-x4qh krb5kdc[18080](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992279, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 04:57:59.658339 9601 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260806 04:57:59.663354 18863 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 04:57:59.660985 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47302 (local address 127.9.96.65:40763)
0806 04:57:59.661644 (+ 659us) server_negotiation.cc:206] Beginning negotiation
0806 04:57:59.661651 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 04:57:59.661677 (+ 26us) server_negotiation.cc:407] Connection header received
0806 04:57:59.661737 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:57:59.661743 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:57:59.661945 (+ 202us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:57:59.662069 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:57:59.662933 (+ 864us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47302: BlockingRecv error: recv got EOF from 127.0.0.1:47302 (error 108)
Metrics: {"server-negotiator.queue_time_us":515,"thread_start_us":357,"threads_started":1}
W20260806 04:57:59.663362 9601 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785992227050689-9601-0/krb5kdc/krb5cc' of type FILE
Aug 06 04:57:59 dist-test-slave-x4qh krb5kdc[18080](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992279, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 04:57:59.664983 9601 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 06 04:57:59 dist-test-slave-x4qh krb5kdc[18080](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992279, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 04:57:59.666285 9601 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 06 04:57:59 dist-test-slave-x4qh krb5kdc[18080](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992279, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.96.65@KRBTEST.COM
I20260806 04:57:59.681506 18863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:57:59.667725 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47314 (local address 127.9.96.65:40763)
0806 04:57:59.667936 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0806 04:57:59.667944 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 04:57:59.668253 (+ 309us) server_negotiation.cc:407] Connection header received
0806 04:57:59.668450 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:57:59.668455 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:57:59.668524 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:57:59.668621 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:57:59.669669 (+ 1048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:57:59.670636 (+ 967us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:57:59.671965 (+ 1329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:57:59.672520 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:57:59.672651 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:57:59.676099 (+ 3448us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:57:59.676117 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:57:59.676131 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:57:59.676161 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:57:59.678774 (+ 2613us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:57:59.679406 (+ 632us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:57:59.679415 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:57:59.679419 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:57:59.679542 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:57:59.679945 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:57:59.679950 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:57:59.679957 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:57:59.680316 (+ 359us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:57:59.680623 (+ 307us) server_negotiation.cc:1035] Waiting for connection context
0806 04:57:59.680949 (+ 326us) server_negotiation.cc:299] Negotiation successful
0806 04:57:59.681212 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
W20260806 04:57:59.686962 18863 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 04:57:59.685303 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47326 (local address 127.9.96.65:40763)
0806 04:57:59.685499 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0806 04:57:59.685507 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 04:57:59.685702 (+ 195us) server_negotiation.cc:407] Connection header received
0806 04:57:59.685874 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:57:59.685878 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:57:59.685928 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:57:59.686018 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:57:59.686738 (+ 720us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47326: BlockingRecv error: recv got EOF from 127.0.0.1:47326 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
W20260806 04:57:59.687191 9601 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785992227050689-9601-0/krb5kdc/krb5cc' of type FILE
Aug 06 04:57:59 dist-test-slave-x4qh krb5kdc[18080](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992279, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 04:57:59.688871 9601 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 06 04:57:59 dist-test-slave-x4qh krb5kdc[18080](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992279, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 04:57:59.690110 9601 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 06 04:57:59 dist-test-slave-x4qh krb5kdc[18080](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992279, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.96.65@KRBTEST.COM
I20260806 04:57:59.702618 18863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:57:59.691249 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47340 (local address 127.9.96.65:40763)
0806 04:57:59.691469 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0806 04:57:59.691478 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 04:57:59.691732 (+ 254us) server_negotiation.cc:407] Connection header received
0806 04:57:59.691843 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:57:59.691846 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:57:59.691892 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:57:59.691963 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:57:59.692779 (+ 816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:57:59.693423 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:57:59.694562 (+ 1139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:57:59.694974 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:57:59.695071 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:57:59.698292 (+ 3221us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:57:59.698298 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:57:59.698302 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:57:59.698336 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:57:59.700627 (+ 2291us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:57:59.701165 (+ 538us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:57:59.701168 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:57:59.701170 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:57:59.701281 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:57:59.701582 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:57:59.701585 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:57:59.701588 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:57:59.701825 (+ 237us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:57:59.701998 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0806 04:57:59.702204 (+ 206us) server_negotiation.cc:299] Negotiation successful
0806 04:57:59.702384 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
W20260806 04:57:59.706453 18863 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 04:57:59.704906 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47344 (local address 127.9.96.65:40763)
0806 04:57:59.705093 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0806 04:57:59.705098 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 04:57:59.705273 (+ 175us) server_negotiation.cc:407] Connection header received
0806 04:57:59.705389 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:57:59.705392 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:57:59.705453 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:57:59.705537 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:57:59.706191 (+ 654us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47344: BlockingRecv error: recv got EOF from 127.0.0.1:47344 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
W20260806 04:57:59.706898 9601 init.cc:244] Error while opening credential cache '/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785992227050689-9601-0/krb5kdc/krb5cc' of type FILE: Runtime error: End of credential cache reached (filename: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785992227050689-9601-0/krb5kdc/krb5cc)
Aug 06 04:57:59 dist-test-slave-x4qh krb5kdc[18080](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992279, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 04:57:59.708325 9601 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 06 04:57:59 dist-test-slave-x4qh krb5kdc[18080](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992279, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 04:57:59.709743 9601 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 06 04:57:59 dist-test-slave-x4qh krb5kdc[18080](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992279, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.96.65@KRBTEST.COM
I20260806 04:57:59.722415 18863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:57:59.710926 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47352 (local address 127.9.96.65:40763)
0806 04:57:59.711107 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0806 04:57:59.711113 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 04:57:59.711232 (+ 119us) server_negotiation.cc:407] Connection header received
0806 04:57:59.711336 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:57:59.711339 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:57:59.711459 (+ 120us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:57:59.711534 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:57:59.712405 (+ 871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:57:59.713076 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:57:59.714176 (+ 1100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:57:59.714560 (+ 384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:57:59.714645 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:57:59.717787 (+ 3142us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:57:59.717792 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:57:59.717795 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:57:59.717823 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:57:59.720154 (+ 2331us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:57:59.720713 (+ 559us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:57:59.720716 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:57:59.720718 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:57:59.720835 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:57:59.721168 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:57:59.721171 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:57:59.721173 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:57:59.721421 (+ 248us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:57:59.721669 (+ 248us) server_negotiation.cc:1035] Waiting for connection context
0806 04:57:59.721896 (+ 227us) server_negotiation.cc:299] Negotiation successful
0806 04:57:59.722084 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260806 04:57:59.723868 9601 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu pid 18239
I20260806 04:57:59.734592 18863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:57:59.724585 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47368 (local address 127.9.96.65:40763)
0806 04:57:59.724787 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0806 04:57:59.724812 (+ 25us) server_negotiation.cc:399] Waiting for connection header
0806 04:57:59.725210 (+ 398us) server_negotiation.cc:407] Connection header received
0806 04:57:59.725333 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:57:59.725337 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:57:59.725393 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:57:59.725460 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:57:59.726269 (+ 809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:57:59.726877 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:57:59.728031 (+ 1154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:57:59.728439 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:57:59.728529 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:57:59.730278 (+ 1749us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:57:59.730282 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:57:59.730285 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:57:59.730311 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:57:59.732653 (+ 2342us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:57:59.733162 (+ 509us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:57:59.733165 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:57:59.733167 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:57:59.733287 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:57:59.733560 (+ 273us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:57:59.733563 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:57:59.733565 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:57:59.733796 (+ 231us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:57:59.733981 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0806 04:57:59.734195 (+ 214us) server_negotiation.cc:299] Negotiation successful
0806 04:57:59.734344 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260806 04:57:59.735178 18435 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 04:57:59.817785 9601 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu with pid 18239
I20260806 04:57:59.842614 9601 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu pid 18452
Aug 06 04:57:59 dist-test-slave-x4qh krb5kdc[18080](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992279, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.96.66@KRBTEST.COM
I20260806 04:57:59.858017 18885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:57:59.843605 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36598 (local address 127.9.96.66:37275)
0806 04:57:59.844111 (+ 506us) server_negotiation.cc:206] Beginning negotiation
0806 04:57:59.844117 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 04:57:59.844142 (+ 25us) server_negotiation.cc:407] Connection header received
0806 04:57:59.844201 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:57:59.844207 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:57:59.844445 (+ 238us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:57:59.844573 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:57:59.845652 (+ 1079us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:57:59.846609 (+ 957us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:57:59.847920 (+ 1311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:57:59.848586 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:57:59.848688 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:57:59.852648 (+ 3960us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:57:59.852658 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:57:59.852671 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:57:59.852700 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:57:59.855633 (+ 2933us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:57:59.856219 (+ 586us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:57:59.856226 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:57:59.856228 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:57:59.856356 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:57:59.856684 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:57:59.856687 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:57:59.856689 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:57:59.857012 (+ 323us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:57:59.857232 (+ 220us) server_negotiation.cc:1035] Waiting for connection context
0806 04:57:59.857471 (+ 239us) server_negotiation.cc:299] Negotiation successful
0806 04:57:59.857722 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":355,"thread_start_us":243,"threads_started":1}
I20260806 04:57:59.859211 18641 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 04:57:59.939492 9601 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu with pid 18452
I20260806 04:57:59.964429 9601 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu pid 18653
Aug 06 04:57:59 dist-test-slave-x4qh krb5kdc[18080](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992279, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.96.67@KRBTEST.COM
I20260806 04:57:59.979741 18888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:57:59.965348 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48050 (local address 127.9.96.67:41109)
0806 04:57:59.966023 (+ 675us) server_negotiation.cc:206] Beginning negotiation
0806 04:57:59.966029 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 04:57:59.966059 (+ 30us) server_negotiation.cc:407] Connection header received
0806 04:57:59.966121 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:57:59.966128 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:57:59.966334 (+ 206us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:57:59.966513 (+ 179us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:57:59.967734 (+ 1221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:57:59.968792 (+ 1058us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:57:59.970001 (+ 1209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:57:59.970568 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:57:59.970660 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:57:59.974514 (+ 3854us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:57:59.974525 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:57:59.974538 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:57:59.974569 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:57:59.977336 (+ 2767us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:57:59.977923 (+ 587us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:57:59.977930 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:57:59.977932 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:57:59.978043 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:57:59.978344 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:57:59.978347 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:57:59.978349 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:57:59.978674 (+ 325us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:57:59.978908 (+ 234us) server_negotiation.cc:1035] Waiting for connection context
0806 04:57:59.979111 (+ 203us) server_negotiation.cc:299] Negotiation successful
0806 04:57:59.979417 (+ 306us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":531,"thread_start_us":352,"threads_started":1}
I20260806 04:57:59.981004 18846 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 04:58:00.068317 9601 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu with pid 18653
I20260806 04:58:00.091672 9601 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu pid 18098
I20260806 04:58:00.092865 18197 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 04:58:00.195276 9601 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu with pid 18098
2026-08-06T04:58:00Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (4100 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785992227050689-9601-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 04:58:00 dist-test-slave-x4qh krb5kdc[18896](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 04:58:00 dist-test-slave-x4qh krb5kdc[18896](info): set up 2 sockets
Aug 06 04:58:00 dist-test-slave-x4qh krb5kdc[18896](info): commencing operation
krb5kdc: starting...
W20260806 04:58:01.475976 9601 mini_kdc.cc:121] Time spent starting KDC: real 1.234s user 0.006s sys 0.019s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785992227050689-9601-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785992227050689-9601-0/krb5kdc/test-user.keytab.
Aug 06 04:58:01 dist-test-slave-x4qh krb5kdc[18896](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992281, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T04:58:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T04:58:01Z Disabled control of system clock
WARNING: no policy specified for oryx/127.9.96.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.9.96.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.9.96.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785992227050689-9601-0/krb5kdc/oryx_127.9.96.126.keytab.
Entry for principal oryx/127.9.96.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785992227050689-9601-0/krb5kdc/oryx_127.9.96.126.keytab.
I20260806 04:58:01.706493 9601 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
/tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785992227050689-9601-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785992227050689-9601-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785992227050689-9601-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785992227050689-9601-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.96.126:40031
--webserver_interface=127.9.96.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785992227050689-9601-0/krb5kdc/oryx_127.9.96.126.keytab
--principal=oryx/127.9.96.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.96.84:34171
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.96.126:40031
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785992227050689-9601-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785992227050689-9601-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785992227050689-9601-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 04:58:02.025043 18915 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 04:58:02.025339 18915 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 04:58:02.025516 18915 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 04:58:02.032687 18915 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 04:58:02.032792 18915 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 04:58:02.032894 18915 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 04:58:02.032995 18915 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 04:58:02.033074 18915 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 04:58:02.042816 18915 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.96.84:34171
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785992227050689-9601-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785992227050689-9601-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.96.126:40031
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785992227050689-9601-0/krb5kdc/oryx_127.9.96.126.keytab
--principal=oryx/127.9.96.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.96.126:40031
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785992227050689-9601-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.96.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785992227050689-9601-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 03:56:12 UTC on ccycloud.quasar-guijbm.root.comops.site
build id 4569618
ASAN enabled
I20260806 04:58:02.044152 18915 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 04:58:02.045439 18915 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 04:58:02.057124 18921 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:58:02.057152 18925 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:58:02.057053 18922 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:58:02.058159 18915 server_base.cc:1034] running on GCE node
I20260806 04:58:02.058951 18915 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 04:58:02.060324 18915 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 04:58:02.061528 18915 hybrid_clock.cc:648] HybridClock initialized: now 1785992282061478 us; error 35 us; skew 500 ppm
I20260806 04:58:02.061883 18915 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 04:58:02 dist-test-slave-x4qh krb5kdc[18896](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992282, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.96.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 04:58:02.064949 18915 init.cc:377] Logged in from keytab as oryx/127.9.96.126@KRBTEST.COM (short username oryx)
I20260806 04:58:02.066416 18915 webserver.cc:460] Webserver started at http://127.9.96.126:46387/ using document root <none> and password file <none>
I20260806 04:58:02.067088 18915 fs_manager.cc:359] Metadata directory not provided
I20260806 04:58:02.067235 18915 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 04:58:02.067589 18915 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 04:58:02.070577 18915 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785992227050689-9601-0/minicluster-data/master-0/data/instance:
uuid: "6126c11fed1e4765becce7bd422ce904"
format_stamp: "Formatted at 2026-08-06 04:58:02 on dist-test-slave-x4qh"
server_key: "31077285de3e067128455876187d67c8"
server_key_iv: "eeb337a4fbcf1bd065ae47b9b0216c65"
server_key_version: "encryptionkey@0"
I20260806 04:58:02.071357 18915 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785992227050689-9601-0/minicluster-data/master-0/wal/instance:
uuid: "6126c11fed1e4765becce7bd422ce904"
format_stamp: "Formatted at 2026-08-06 04:58:02 on dist-test-slave-x4qh"
server_key: "31077285de3e067128455876187d67c8"
server_key_iv: "eeb337a4fbcf1bd065ae47b9b0216c65"
server_key_version: "encryptionkey@0"
I20260806 04:58:02.076392 18915 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260806 04:58:02.079717 18934 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 04:58:02.081007 18915 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260806 04:58:02.081187 18915 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785992227050689-9601-0/minicluster-data/master-0/data,/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785992227050689-9601-0/minicluster-data/master-0/wal
uuid: "6126c11fed1e4765becce7bd422ce904"
format_stamp: "Formatted at 2026-08-06 04:58:02 on dist-test-slave-x4qh"
server_key: "31077285de3e067128455876187d67c8"
server_key_iv: "eeb337a4fbcf1bd065ae47b9b0216c65"
server_key_version: "encryptionkey@0"
I20260806 04:58:02.081372 18915 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785992227050689-9601-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785992227050689-9601-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785992227050689-9601-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 04:58:02.097718 18915 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 04:58:02.101578 18915 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 04:58:02.101868 18915 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 04:58:02.120142 18915 rpc_server.cc:307] RPC server started. Bound to: 127.9.96.126:40031
I20260806 04:58:02.120186 19007 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.96.126:40031 every 8 connection(s)
I20260806 04:58:02.121411 18915 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785992227050689-9601-0/minicluster-data/master-0/data/info.pb
I20260806 04:58:02.124866 19009 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 04:58:02.124886 9601 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu as pid 18915
I20260806 04:58:02.125100 9601 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785992227050689-9601-0/minicluster-data/master-0/wal/instance
I20260806 04:58:02.125782 9601 external_mini_cluster.cc:1428] Setting key 1b2d58aff4142c5b026f725c32574de2
I20260806 04:58:02.138696 19009 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6126c11fed1e4765becce7bd422ce904: Bootstrap starting.
Aug 06 04:58:02 dist-test-slave-x4qh krb5kdc[18896](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992281, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.9.96.126@KRBTEST.COM
I20260806 04:58:02.141757 19009 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6126c11fed1e4765becce7bd422ce904: Neither blocks nor log segments found. Creating new log.
I20260806 04:58:02.143009 19009 log.cc:824] T 00000000000000000000000000000000 P 6126c11fed1e4765becce7bd422ce904: Log is configured to *not* fsync() on all Append() calls
I20260806 04:58:02.145820 19009 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6126c11fed1e4765becce7bd422ce904: No bootstrap required, opened a new log
I20260806 04:58:02.149137 19014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:58:02.127686 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48702 (local address 127.9.96.126:40031)
0806 04:58:02.128507 (+ 821us) server_negotiation.cc:206] Beginning negotiation
0806 04:58:02.128524 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0806 04:58:02.128571 (+ 47us) server_negotiation.cc:407] Connection header received
0806 04:58:02.131334 (+ 2763us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:58:02.131362 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:58:02.131840 (+ 478us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:58:02.132311 (+ 471us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:58:02.133578 (+ 1267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:02.135611 (+ 2033us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:02.136770 (+ 1159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:02.137745 (+ 975us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:02.137925 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:58:02.141833 (+ 3908us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:58:02.141847 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:58:02.141880 (+ 33us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:58:02.141937 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:58:02.145161 (+ 3224us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:58:02.145848 (+ 687us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:58:02.145855 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:58:02.145865 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:58:02.146065 (+ 200us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:58:02.146443 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:58:02.146449 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:58:02.146452 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:58:02.146965 (+ 513us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:58:02.147276 (+ 311us) server_negotiation.cc:1035] Waiting for connection context
0806 04:58:02.147754 (+ 478us) server_negotiation.cc:299] Negotiation successful
0806 04:58:02.148168 (+ 414us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":593,"thread_start_us":344,"threads_started":1}
I20260806 04:58:02.153654 19009 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6126c11fed1e4765becce7bd422ce904 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6126c11fed1e4765becce7bd422ce904" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 40031 } }
I20260806 04:58:02.154008 19009 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6126c11fed1e4765becce7bd422ce904 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 04:58:02.154131 19009 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6126c11fed1e4765becce7bd422ce904 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6126c11fed1e4765becce7bd422ce904, State: Initialized, Role: FOLLOWER
I20260806 04:58:02.154685 19009 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6126c11fed1e4765becce7bd422ce904 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6126c11fed1e4765becce7bd422ce904" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 40031 } }
I20260806 04:58:02.154930 19009 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6126c11fed1e4765becce7bd422ce904 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 04:58:02.155108 19009 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6126c11fed1e4765becce7bd422ce904 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 04:58:02.155280 19009 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6126c11fed1e4765becce7bd422ce904 [term 0 FOLLOWER]: Advancing to term 1
I20260806 04:58:02.157218 19009 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6126c11fed1e4765becce7bd422ce904 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6126c11fed1e4765becce7bd422ce904" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 40031 } }
I20260806 04:58:02.157707 19009 leader_election.cc:302] T 00000000000000000000000000000000 P 6126c11fed1e4765becce7bd422ce904 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6126c11fed1e4765becce7bd422ce904; no voters:
I20260806 04:58:02.158317 19009 leader_election.cc:288] T 00000000000000000000000000000000 P 6126c11fed1e4765becce7bd422ce904 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 04:58:02.158504 19016 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6126c11fed1e4765becce7bd422ce904 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 04:58:02.159196 19016 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6126c11fed1e4765becce7bd422ce904 [term 1 LEADER]: Becoming Leader. State: Replica: 6126c11fed1e4765becce7bd422ce904, State: Running, Role: LEADER
I20260806 04:58:02.159824 19016 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6126c11fed1e4765becce7bd422ce904 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6126c11fed1e4765becce7bd422ce904" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 40031 } }
I20260806 04:58:02.160434 19009 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6126c11fed1e4765becce7bd422ce904 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 04:58:02.162683 19017 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6126c11fed1e4765becce7bd422ce904 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6126c11fed1e4765becce7bd422ce904" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6126c11fed1e4765becce7bd422ce904" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 40031 } } }
I20260806 04:58:02.162658 19018 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6126c11fed1e4765becce7bd422ce904 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6126c11fed1e4765becce7bd422ce904. Latest consensus state: current_term: 1 leader_uuid: "6126c11fed1e4765becce7bd422ce904" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6126c11fed1e4765becce7bd422ce904" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 40031 } } }
I20260806 04:58:02.163055 19018 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6126c11fed1e4765becce7bd422ce904 [sys.catalog]: This master's current role is: LEADER
I20260806 04:58:02.163051 19017 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6126c11fed1e4765becce7bd422ce904 [sys.catalog]: This master's current role is: LEADER
I20260806 04:58:02.164832 19024 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 04:58:02.170388 19024 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 04:58:02.178593 19024 catalog_manager.cc:1347] Generated new cluster ID: b1cd49f4c4814e84b0f51af05de8eeee
I20260806 04:58:02.178701 19024 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 04:58:02.185782 19024 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 04:58:02.187202 19024 catalog_manager.cc:1504] Loading token signing keys...
I20260806 04:58:02.193666 19024 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6126c11fed1e4765becce7bd422ce904: Generated new TSK 0
I20260806 04:58:02.194351 19024 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.9.96.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.9.96.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.9.96.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785992227050689-9601-0/krb5kdc/oryx_127.9.96.65.keytab.
Entry for principal oryx/127.9.96.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785992227050689-9601-0/krb5kdc/oryx_127.9.96.65.keytab.
I20260806 04:58:02.254662 9601 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
/tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785992227050689-9601-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785992227050689-9601-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785992227050689-9601-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785992227050689-9601-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.96.65:0
--local_ip_for_outbound_sockets=127.9.96.65
--webserver_interface=127.9.96.65
--webserver_port=0
--tserver_master_addrs=127.9.96.126:40031
--keytab_file=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785992227050689-9601-0/krb5kdc/oryx_127.9.96.65.keytab
--principal=oryx/127.9.96.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.96.84:34171
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785992227050689-9601-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785992227050689-9601-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785992227050689-9601-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 04:58:02.577955 19048 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 04:58:02.578250 19048 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 04:58:02.578418 19048 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 04:58:02.585690 19048 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 04:58:02.585825 19048 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 04:58:02.585992 19048 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.96.65
I20260806 04:58:02.595633 19048 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.96.84:34171
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785992227050689-9601-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785992227050689-9601-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785992227050689-9601-0/krb5kdc/oryx_127.9.96.65.keytab
--principal=oryx/127.9.96.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.96.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785992227050689-9601-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.96.65
--webserver_port=0
--tserver_master_addrs=127.9.96.126:40031
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.96.65
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785992227050689-9601-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 03:56:12 UTC on ccycloud.quasar-guijbm.root.comops.site
build id 4569618
ASAN enabled
I20260806 04:58:02.596876 19048 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 04:58:02.598178 19048 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 04:58:02.611112 19054 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:58:02.611168 19060 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:58:02.611210 19058 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:58:02.612082 19048 server_base.cc:1034] running on GCE node
I20260806 04:58:02.612711 19048 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 04:58:02.613571 19048 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 04:58:02.614813 19048 hybrid_clock.cc:648] HybridClock initialized: now 1785992282614755 us; error 49 us; skew 500 ppm
I20260806 04:58:02.615111 19048 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 04:58:02 dist-test-slave-x4qh krb5kdc[18896](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992282, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.96.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 04:58:02.618345 19048 init.cc:377] Logged in from keytab as oryx/127.9.96.65@KRBTEST.COM (short username oryx)
I20260806 04:58:02.619737 19048 webserver.cc:460] Webserver started at http://127.9.96.65:37575/ using document root <none> and password file <none>
I20260806 04:58:02.620358 19048 fs_manager.cc:359] Metadata directory not provided
I20260806 04:58:02.620456 19048 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 04:58:02.620705 19048 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 04:58:02.623461 19048 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785992227050689-9601-0/minicluster-data/ts-0/data/instance:
uuid: "2259df3ac9ca47bfb61228d968d524fd"
format_stamp: "Formatted at 2026-08-06 04:58:02 on dist-test-slave-x4qh"
server_key: "d04f51fa2c3f6878369102bba77c9d82"
server_key_iv: "1e1993b305485b2d8c5822cb57678502"
server_key_version: "encryptionkey@0"
I20260806 04:58:02.624198 19048 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785992227050689-9601-0/minicluster-data/ts-0/wal/instance:
uuid: "2259df3ac9ca47bfb61228d968d524fd"
format_stamp: "Formatted at 2026-08-06 04:58:02 on dist-test-slave-x4qh"
server_key: "d04f51fa2c3f6878369102bba77c9d82"
server_key_iv: "1e1993b305485b2d8c5822cb57678502"
server_key_version: "encryptionkey@0"
I20260806 04:58:02.629132 19048 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260806 04:58:02.632362 19069 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 04:58:02.633556 19048 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260806 04:58:02.633728 19048 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785992227050689-9601-0/minicluster-data/ts-0/data,/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785992227050689-9601-0/minicluster-data/ts-0/wal
uuid: "2259df3ac9ca47bfb61228d968d524fd"
format_stamp: "Formatted at 2026-08-06 04:58:02 on dist-test-slave-x4qh"
server_key: "d04f51fa2c3f6878369102bba77c9d82"
server_key_iv: "1e1993b305485b2d8c5822cb57678502"
server_key_version: "encryptionkey@0"
I20260806 04:58:02.633924 19048 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785992227050689-9601-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785992227050689-9601-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785992227050689-9601-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 04:58:02.656663 19048 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 04:58:02.660972 19048 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 04:58:02.661250 19048 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 04:58:02.662236 19048 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 04:58:02.663551 19048 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 04:58:02.663885 19048 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 04:58:02.664040 19048 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 04:58:02.664112 19048 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 04:58:02.697561 19048 rpc_server.cc:307] RPC server started. Bound to: 127.9.96.65:37807
I20260806 04:58:02.697645 19232 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.96.65:37807 every 8 connection(s)
I20260806 04:58:02.698884 19048 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785992227050689-9601-0/minicluster-data/ts-0/data/info.pb
I20260806 04:58:02.705772 9601 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu as pid 19048
I20260806 04:58:02.706043 9601 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785992227050689-9601-0/minicluster-data/ts-0/wal/instance
I20260806 04:58:02.706858 9601 external_mini_cluster.cc:1428] Setting key fa657bd0061542521cbb28918d56b7a8
Aug 06 04:58:02 dist-test-slave-x4qh krb5kdc[18896](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992282, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.96.65@KRBTEST.COM for oryx/127.9.96.126@KRBTEST.COM
I20260806 04:58:02.719952 19236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:58:02.701411 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.96.65:33211 (local address 127.9.96.126:40031)
0806 04:58:02.701862 (+ 451us) server_negotiation.cc:206] Beginning negotiation
0806 04:58:02.701866 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 04:58:02.702684 (+ 818us) server_negotiation.cc:407] Connection header received
0806 04:58:02.704584 (+ 1900us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:58:02.704591 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:58:02.704691 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:58:02.704815 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:58:02.706729 (+ 1914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:02.707648 (+ 919us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:02.709672 (+ 2024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:02.710068 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:02.710166 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:58:02.714288 (+ 4122us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:58:02.714293 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:58:02.714297 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:58:02.714329 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:58:02.716971 (+ 2642us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:58:02.717668 (+ 697us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:58:02.717675 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:58:02.717678 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:58:02.717814 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:58:02.718244 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:58:02.718250 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:58:02.718253 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:58:02.718515 (+ 262us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:58:02.718670 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0806 04:58:02.719337 (+ 667us) server_negotiation.cc:299] Negotiation successful
0806 04:58:02.719594 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":342,"thread_start_us":251,"threads_started":1}
I20260806 04:58:02.720839 19237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:58:02.701796 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:40031 (local address 127.9.96.65:33211)
0806 04:58:02.702529 (+ 733us) negotiation.cc:107] Waiting for socket to connect
0806 04:58:02.702569 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0806 04:58:02.704312 (+ 1743us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:58:02.705028 (+ 716us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:58:02.705036 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:58:02.705724 (+ 688us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:58:02.706531 (+ 807us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:58:02.706564 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:58:02.707824 (+ 1260us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:58:02.707829 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:58:02.709509 (+ 1680us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:58:02.709528 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:58:02.710291 (+ 763us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:58:02.710310 (+ 19us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:58:02.710632 (+ 322us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:58:02.711354 (+ 722us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:58:02.711404 (+ 50us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:58:02.714059 (+ 2655us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 04:58:02.717112 (+ 3053us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:58:02.717121 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:58:02.717136 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:58:02.717549 (+ 413us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:58:02.717925 (+ 376us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:58:02.717930 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:58:02.717933 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:58:02.718132 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:58:02.718647 (+ 515us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 04:58:02.718654 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 04:58:02.718969 (+ 315us) client_negotiation.cc:769] Sending connection context
0806 04:58:02.719236 (+ 267us) client_negotiation.cc:240] Negotiation successful
0806 04:58:02.719744 (+ 508us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":474,"thread_start_us":266,"threads_started":1}
I20260806 04:58:02.722369 19234 heartbeater.cc:347] Connected to a master server at 127.9.96.126:40031
I20260806 04:58:02.722692 19234 heartbeater.cc:464] Registering TS with master...
I20260806 04:58:02.723564 19234 heartbeater.cc:511] Master 127.9.96.126:40031 requested a full tablet report, sending...
I20260806 04:58:02.725953 18960 ts_manager.cc:194] Registered new tserver with Master: 2259df3ac9ca47bfb61228d968d524fd (127.9.96.65:37807)
I20260806 04:58:02.728256 18960 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.9.96.65@KRBTEST.COM'} at 127.9.96.65:33211
WARNING: no policy specified for oryx/127.9.96.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.9.96.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.9.96.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785992227050689-9601-0/krb5kdc/oryx_127.9.96.66.keytab.
Entry for principal oryx/127.9.96.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785992227050689-9601-0/krb5kdc/oryx_127.9.96.66.keytab.
I20260806 04:58:02.762768 9601 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
/tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785992227050689-9601-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785992227050689-9601-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785992227050689-9601-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785992227050689-9601-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.96.66:0
--local_ip_for_outbound_sockets=127.9.96.66
--webserver_interface=127.9.96.66
--webserver_port=0
--tserver_master_addrs=127.9.96.126:40031
--keytab_file=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785992227050689-9601-0/krb5kdc/oryx_127.9.96.66.keytab
--principal=oryx/127.9.96.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.96.84:34171
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785992227050689-9601-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785992227050689-9601-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785992227050689-9601-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 04:58:03.086285 19240 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 04:58:03.086544 19240 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 04:58:03.086632 19240 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 04:58:03.093303 19240 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 04:58:03.093396 19240 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 04:58:03.093508 19240 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.96.66
I20260806 04:58:03.102802 19240 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.96.84:34171
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785992227050689-9601-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785992227050689-9601-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785992227050689-9601-0/krb5kdc/oryx_127.9.96.66.keytab
--principal=oryx/127.9.96.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.96.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785992227050689-9601-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.96.66
--webserver_port=0
--tserver_master_addrs=127.9.96.126:40031
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.96.66
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785992227050689-9601-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 03:56:12 UTC on ccycloud.quasar-guijbm.root.comops.site
build id 4569618
ASAN enabled
I20260806 04:58:03.103922 19240 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 04:58:03.105163 19240 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 04:58:03.117781 19250 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:58:03.117740 19253 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:58:03.117847 19251 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:58:03.118942 19240 server_base.cc:1034] running on GCE node
I20260806 04:58:03.119655 19240 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 04:58:03.120723 19240 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 04:58:03.121978 19240 hybrid_clock.cc:648] HybridClock initialized: now 1785992283121898 us; error 85 us; skew 500 ppm
I20260806 04:58:03.122303 19240 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 04:58:03 dist-test-slave-x4qh krb5kdc[18896](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992283, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.96.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 04:58:03.125978 19240 init.cc:377] Logged in from keytab as oryx/127.9.96.66@KRBTEST.COM (short username oryx)
I20260806 04:58:03.127449 19240 webserver.cc:460] Webserver started at http://127.9.96.66:45523/ using document root <none> and password file <none>
I20260806 04:58:03.128067 19240 fs_manager.cc:359] Metadata directory not provided
I20260806 04:58:03.128185 19240 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 04:58:03.128502 19240 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 04:58:03.131242 19240 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785992227050689-9601-0/minicluster-data/ts-1/data/instance:
uuid: "50ea117b4ffa45bc8456eb9ec24594b3"
format_stamp: "Formatted at 2026-08-06 04:58:03 on dist-test-slave-x4qh"
server_key: "6be8ccac887ffa4d995fbf54c2a43266"
server_key_iv: "def13310c91e1f0809786561febe5862"
server_key_version: "encryptionkey@0"
I20260806 04:58:03.132102 19240 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785992227050689-9601-0/minicluster-data/ts-1/wal/instance:
uuid: "50ea117b4ffa45bc8456eb9ec24594b3"
format_stamp: "Formatted at 2026-08-06 04:58:03 on dist-test-slave-x4qh"
server_key: "6be8ccac887ffa4d995fbf54c2a43266"
server_key_iv: "def13310c91e1f0809786561febe5862"
server_key_version: "encryptionkey@0"
I20260806 04:58:03.137109 19240 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260806 04:58:03.140518 19260 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 04:58:03.141752 19240 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 04:58:03.141933 19240 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785992227050689-9601-0/minicluster-data/ts-1/data,/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785992227050689-9601-0/minicluster-data/ts-1/wal
uuid: "50ea117b4ffa45bc8456eb9ec24594b3"
format_stamp: "Formatted at 2026-08-06 04:58:03 on dist-test-slave-x4qh"
server_key: "6be8ccac887ffa4d995fbf54c2a43266"
server_key_iv: "def13310c91e1f0809786561febe5862"
server_key_version: "encryptionkey@0"
I20260806 04:58:03.142128 19240 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785992227050689-9601-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785992227050689-9601-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785992227050689-9601-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 04:58:03.158960 19240 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 04:58:03.163131 19240 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 04:58:03.163466 19240 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 04:58:03.164345 19240 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 04:58:03.165793 19240 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 04:58:03.165868 19240 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 04:58:03.166064 19240 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 04:58:03.166149 19240 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 04:58:03.197926 19240 rpc_server.cc:307] RPC server started. Bound to: 127.9.96.66:41607
I20260806 04:58:03.198009 19430 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.96.66:41607 every 8 connection(s)
I20260806 04:58:03.199174 19240 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785992227050689-9601-0/minicluster-data/ts-1/data/info.pb
I20260806 04:58:03.204268 9601 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu as pid 19240
I20260806 04:58:03.204494 9601 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785992227050689-9601-0/minicluster-data/ts-1/wal/instance
I20260806 04:58:03.204943 9601 external_mini_cluster.cc:1428] Setting key 41c2e686a255d067b375957ee88e184c
Aug 06 04:58:03 dist-test-slave-x4qh krb5kdc[18896](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992283, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.96.66@KRBTEST.COM for oryx/127.9.96.126@KRBTEST.COM
I20260806 04:58:03.219973 19236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:58:03.201662 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.96.66:57163 (local address 127.9.96.126:40031)
0806 04:58:03.201838 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0806 04:58:03.201843 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 04:58:03.202903 (+ 1060us) server_negotiation.cc:407] Connection header received
0806 04:58:03.204815 (+ 1912us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:58:03.204822 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:58:03.204906 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:58:03.205013 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:58:03.206814 (+ 1801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:03.207856 (+ 1042us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:03.209369 (+ 1513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:03.210118 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:03.210259 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:58:03.213771 (+ 3512us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:58:03.213781 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:58:03.213789 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:58:03.213837 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:58:03.216982 (+ 3145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:58:03.217754 (+ 772us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:58:03.217758 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:58:03.217760 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:58:03.217884 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:58:03.218338 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:58:03.218341 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:58:03.218343 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:58:03.218594 (+ 251us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:58:03.218730 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0806 04:58:03.219557 (+ 827us) server_negotiation.cc:299] Negotiation successful
0806 04:58:03.219729 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260806 04:58:03.221436 19434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:58:03.202096 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:40031 (local address 127.9.96.66:57163)
0806 04:58:03.202753 (+ 657us) negotiation.cc:107] Waiting for socket to connect
0806 04:58:03.202798 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0806 04:58:03.204525 (+ 1727us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:58:03.205241 (+ 716us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:58:03.205248 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:58:03.205867 (+ 619us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:58:03.206635 (+ 768us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:58:03.206676 (+ 41us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:58:03.208010 (+ 1334us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:58:03.208014 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:58:03.209219 (+ 1205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:58:03.209231 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:58:03.210315 (+ 1084us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:58:03.210320 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:58:03.210504 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:58:03.211141 (+ 637us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:58:03.211161 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:58:03.213531 (+ 2370us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 04:58:03.217131 (+ 3600us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:58:03.217142 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:58:03.217157 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:58:03.217658 (+ 501us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:58:03.217984 (+ 326us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:58:03.217992 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:58:03.217998 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:58:03.218227 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:58:03.218742 (+ 515us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 04:58:03.218752 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 04:58:03.219165 (+ 413us) client_negotiation.cc:769] Sending connection context
0806 04:58:03.219493 (+ 328us) client_negotiation.cc:240] Negotiation successful
0806 04:58:03.220031 (+ 538us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":446,"thread_start_us":249,"threads_started":1}
I20260806 04:58:03.222844 19431 heartbeater.cc:347] Connected to a master server at 127.9.96.126:40031
I20260806 04:58:03.223120 19431 heartbeater.cc:464] Registering TS with master...
I20260806 04:58:03.223845 19431 heartbeater.cc:511] Master 127.9.96.126:40031 requested a full tablet report, sending...
I20260806 04:58:03.225258 18960 ts_manager.cc:194] Registered new tserver with Master: 50ea117b4ffa45bc8456eb9ec24594b3 (127.9.96.66:41607)
I20260806 04:58:03.226053 18960 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.9.96.66@KRBTEST.COM'} at 127.9.96.66:57163
WARNING: no policy specified for oryx/127.9.96.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.9.96.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.9.96.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785992227050689-9601-0/krb5kdc/oryx_127.9.96.67.keytab.
Entry for principal oryx/127.9.96.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785992227050689-9601-0/krb5kdc/oryx_127.9.96.67.keytab.
I20260806 04:58:03.258368 9601 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
/tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785992227050689-9601-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785992227050689-9601-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785992227050689-9601-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785992227050689-9601-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.96.67:0
--local_ip_for_outbound_sockets=127.9.96.67
--webserver_interface=127.9.96.67
--webserver_port=0
--tserver_master_addrs=127.9.96.126:40031
--keytab_file=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785992227050689-9601-0/krb5kdc/oryx_127.9.96.67.keytab
--principal=oryx/127.9.96.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.96.84:34171
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785992227050689-9601-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785992227050689-9601-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785992227050689-9601-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 04:58:03.578943 19439 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 04:58:03.579243 19439 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 04:58:03.579423 19439 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 04:58:03.586661 19439 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 04:58:03.586786 19439 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 04:58:03.586966 19439 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.96.67
I20260806 04:58:03.595957 19439 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.96.84:34171
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785992227050689-9601-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785992227050689-9601-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785992227050689-9601-0/krb5kdc/oryx_127.9.96.67.keytab
--principal=oryx/127.9.96.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.96.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785992227050689-9601-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.96.67
--webserver_port=0
--tserver_master_addrs=127.9.96.126:40031
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.96.67
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785992227050689-9601-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 03:56:12 UTC on ccycloud.quasar-guijbm.root.comops.site
build id 4569618
ASAN enabled
I20260806 04:58:03.597258 19439 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 04:58:03.598579 19439 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 04:58:03.611070 19449 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:58:03.611176 19448 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:58:03.611142 19454 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:58:03.611943 19439 server_base.cc:1034] running on GCE node
I20260806 04:58:03.612500 19439 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 04:58:03.613405 19439 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 04:58:03.614624 19439 hybrid_clock.cc:648] HybridClock initialized: now 1785992283614594 us; error 53 us; skew 500 ppm
I20260806 04:58:03.614920 19439 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 04:58:03 dist-test-slave-x4qh krb5kdc[18896](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992283, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.96.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 04:58:03.617923 19439 init.cc:377] Logged in from keytab as oryx/127.9.96.67@KRBTEST.COM (short username oryx)
I20260806 04:58:03.619232 19439 webserver.cc:460] Webserver started at http://127.9.96.67:38151/ using document root <none> and password file <none>
I20260806 04:58:03.619947 19439 fs_manager.cc:359] Metadata directory not provided
I20260806 04:58:03.620083 19439 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 04:58:03.620374 19439 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 04:58:03.623080 19439 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785992227050689-9601-0/minicluster-data/ts-2/data/instance:
uuid: "aab3b242cd3e451e958340a207168137"
format_stamp: "Formatted at 2026-08-06 04:58:03 on dist-test-slave-x4qh"
server_key: "72dd9f89eb8120ef25689e7681aea634"
server_key_iv: "44939536f15b0f4e2aaaf9d56ce948a9"
server_key_version: "encryptionkey@0"
I20260806 04:58:03.623870 19439 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785992227050689-9601-0/minicluster-data/ts-2/wal/instance:
uuid: "aab3b242cd3e451e958340a207168137"
format_stamp: "Formatted at 2026-08-06 04:58:03 on dist-test-slave-x4qh"
server_key: "72dd9f89eb8120ef25689e7681aea634"
server_key_iv: "44939536f15b0f4e2aaaf9d56ce948a9"
server_key_version: "encryptionkey@0"
I20260806 04:58:03.629159 19439 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260806 04:58:03.632474 19464 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 04:58:03.633687 19439 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260806 04:58:03.633870 19439 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785992227050689-9601-0/minicluster-data/ts-2/data,/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785992227050689-9601-0/minicluster-data/ts-2/wal
uuid: "aab3b242cd3e451e958340a207168137"
format_stamp: "Formatted at 2026-08-06 04:58:03 on dist-test-slave-x4qh"
server_key: "72dd9f89eb8120ef25689e7681aea634"
server_key_iv: "44939536f15b0f4e2aaaf9d56ce948a9"
server_key_version: "encryptionkey@0"
I20260806 04:58:03.634058 19439 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785992227050689-9601-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785992227050689-9601-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785992227050689-9601-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 04:58:03.647203 19439 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 04:58:03.651217 19439 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 04:58:03.651553 19439 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 04:58:03.652443 19439 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 04:58:03.653717 19439 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 04:58:03.653837 19439 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 04:58:03.654011 19439 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 04:58:03.654095 19439 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 04:58:03.685931 19439 rpc_server.cc:307] RPC server started. Bound to: 127.9.96.67:38449
I20260806 04:58:03.685973 19632 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.96.67:38449 every 8 connection(s)
I20260806 04:58:03.687191 19439 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785992227050689-9601-0/minicluster-data/ts-2/data/info.pb
I20260806 04:58:03.688350 9601 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu as pid 19439
I20260806 04:58:03.688570 9601 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785992227050689-9601-0/minicluster-data/ts-2/wal/instance
I20260806 04:58:03.689023 9601 external_mini_cluster.cc:1428] Setting key 58f7b5a3c1ab0ac50f42b45cab848c1e
Aug 06 04:58:03 dist-test-slave-x4qh krb5kdc[18896](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992283, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.96.67@KRBTEST.COM for oryx/127.9.96.126@KRBTEST.COM
I20260806 04:58:03.706513 19236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:58:03.690007 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.96.67:36327 (local address 127.9.96.126:40031)
0806 04:58:03.690213 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0806 04:58:03.690219 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 04:58:03.691684 (+ 1465us) server_negotiation.cc:407] Connection header received
0806 04:58:03.693387 (+ 1703us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:58:03.693391 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:58:03.693488 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:58:03.693568 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:58:03.694957 (+ 1389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:03.695760 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:03.697222 (+ 1462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:03.697614 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:03.697742 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:58:03.701139 (+ 3397us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:58:03.701144 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:58:03.701147 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:58:03.701176 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:58:03.703636 (+ 2460us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:58:03.704311 (+ 675us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:58:03.704318 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:58:03.704321 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:58:03.704463 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:58:03.704888 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:58:03.704893 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:58:03.704897 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:58:03.705171 (+ 274us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:58:03.705326 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0806 04:58:03.705970 (+ 644us) server_negotiation.cc:299] Negotiation successful
0806 04:58:03.706185 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260806 04:58:03.707301 19635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:58:03.690563 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:40031 (local address 127.9.96.67:36327)
0806 04:58:03.691479 (+ 916us) negotiation.cc:107] Waiting for socket to connect
0806 04:58:03.691533 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0806 04:58:03.693156 (+ 1623us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:58:03.693778 (+ 622us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:58:03.693785 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:58:03.694264 (+ 479us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:58:03.694812 (+ 548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:58:03.694828 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:58:03.695882 (+ 1054us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:58:03.695886 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:58:03.697094 (+ 1208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:58:03.697108 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:58:03.697838 (+ 730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:58:03.697845 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:58:03.698083 (+ 238us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:58:03.698753 (+ 670us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:58:03.698775 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:58:03.700936 (+ 2161us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 04:58:03.703780 (+ 2844us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:58:03.703788 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:58:03.703801 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:58:03.704192 (+ 391us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:58:03.704576 (+ 384us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:58:03.704581 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:58:03.704585 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:58:03.704769 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:58:03.705300 (+ 531us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 04:58:03.705307 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 04:58:03.705604 (+ 297us) client_negotiation.cc:769] Sending connection context
0806 04:58:03.705855 (+ 251us) client_negotiation.cc:240] Negotiation successful
0806 04:58:03.706308 (+ 453us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":599,"thread_start_us":375,"threads_started":1}
I20260806 04:58:03.708467 19633 heartbeater.cc:347] Connected to a master server at 127.9.96.126:40031
I20260806 04:58:03.708762 19633 heartbeater.cc:464] Registering TS with master...
I20260806 04:58:03.709429 19633 heartbeater.cc:511] Master 127.9.96.126:40031 requested a full tablet report, sending...
I20260806 04:58:03.710772 18960 ts_manager.cc:194] Registered new tserver with Master: aab3b242cd3e451e958340a207168137 (127.9.96.67:38449)
I20260806 04:58:03.711606 18960 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.9.96.67@KRBTEST.COM'} at 127.9.96.67:36327
I20260806 04:58:03.716415 9601 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 04:58:03.731853 19234 heartbeater.cc:503] Master 127.9.96.126:40031 was elected leader, sending a full tablet report...
WARNING: no policy specified for oryx/127.9.96.125@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.9.96.125@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.9.96.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785992227050689-9601-0/krb5kdc/oryx_127.9.96.125.keytab.
Entry for principal oryx/127.9.96.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785992227050689-9601-0/krb5kdc/oryx_127.9.96.125.keytab.
I20260806 04:58:03.773288 9601 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
/tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785992227050689-9601-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785992227050689-9601-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785992227050689-9601-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785992227050689-9601-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.96.125:39617
--webserver_interface=127.9.96.125
--webserver_port=0
--keytab_file=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785992227050689-9601-0/krb5kdc/oryx_127.9.96.125.keytab
--principal=oryx/127.9.96.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.96.84:34171
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.96.126:40031,127.9.96.125:39617
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785992227050689-9601-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785992227050689-9601-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785992227050689-9601-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 04:58:04.097615 19640 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 04:58:04.097906 19640 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 04:58:04.098073 19640 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 04:58:04.104969 19640 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 04:58:04.105093 19640 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 04:58:04.105180 19640 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 04:58:04.105276 19640 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 04:58:04.105357 19640 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 04:58:04.114897 19640 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.96.84:34171
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785992227050689-9601-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785992227050689-9601-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.9.96.126:40031,127.9.96.125:39617
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785992227050689-9601-0/krb5kdc/oryx_127.9.96.125.keytab
--principal=oryx/127.9.96.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.96.125:39617
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785992227050689-9601-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.9.96.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785992227050689-9601-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 03:56:12 UTC on ccycloud.quasar-guijbm.root.comops.site
build id 4569618
ASAN enabled
W20260806 04:58:04.115242 19640 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.9.96.126:40031,127.9.96.125:39617'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260806 04:58:04.116278 19640 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 04:58:04.117587 19640 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 04:58:04.129062 19658 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:58:04.129114 19651 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:58:04.129144 19653 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:58:04.130012 19640 server_base.cc:1034] running on GCE node
I20260806 04:58:04.130684 19640 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 04:58:04.132042 19640 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 04:58:04.133309 19640 hybrid_clock.cc:648] HybridClock initialized: now 1785992284133249 us; error 50 us; skew 500 ppm
I20260806 04:58:04.133613 19640 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 04:58:04 dist-test-slave-x4qh krb5kdc[18896](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992284, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.96.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 04:58:04.136773 19640 init.cc:377] Logged in from keytab as oryx/127.9.96.125@KRBTEST.COM (short username oryx)
I20260806 04:58:04.138226 19640 webserver.cc:460] Webserver started at http://127.9.96.125:46405/ using document root <none> and password file <none>
I20260806 04:58:04.138988 19640 fs_manager.cc:359] Metadata directory not provided
I20260806 04:58:04.139107 19640 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 04:58:04.139477 19640 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 04:58:04.142431 19640 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785992227050689-9601-0/minicluster-data/master-1/data/instance:
uuid: "89e9df10c70d4ddaacd2a30fff7a3b2d"
format_stamp: "Formatted at 2026-08-06 04:58:04 on dist-test-slave-x4qh"
server_key: "95dff7d38bf45a98e8a13c1e20ce4a30"
server_key_iv: "44aa72342ba02f364935e50cb4c16f16"
server_key_version: "encryptionkey@0"
I20260806 04:58:04.143237 19640 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785992227050689-9601-0/minicluster-data/master-1/wal/instance:
uuid: "89e9df10c70d4ddaacd2a30fff7a3b2d"
format_stamp: "Formatted at 2026-08-06 04:58:04 on dist-test-slave-x4qh"
server_key: "95dff7d38bf45a98e8a13c1e20ce4a30"
server_key_iv: "44aa72342ba02f364935e50cb4c16f16"
server_key_version: "encryptionkey@0"
I20260806 04:58:04.148844 19640 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.002s
I20260806 04:58:04.152217 19668 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 04:58:04.153466 19640 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.004s
I20260806 04:58:04.153635 19640 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785992227050689-9601-0/minicluster-data/master-1/data,/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785992227050689-9601-0/minicluster-data/master-1/wal
uuid: "89e9df10c70d4ddaacd2a30fff7a3b2d"
format_stamp: "Formatted at 2026-08-06 04:58:04 on dist-test-slave-x4qh"
server_key: "95dff7d38bf45a98e8a13c1e20ce4a30"
server_key_iv: "44aa72342ba02f364935e50cb4c16f16"
server_key_version: "encryptionkey@0"
I20260806 04:58:04.153862 19640 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785992227050689-9601-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785992227050689-9601-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785992227050689-9601-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 04:58:04.172674 19640 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 04:58:04.176784 19640 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 04:58:04.177062 19640 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 04:58:04.194980 19640 rpc_server.cc:307] RPC server started. Bound to: 127.9.96.125:39617
I20260806 04:58:04.195011 19750 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.96.125:39617 every 8 connection(s)
I20260806 04:58:04.196383 19640 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785992227050689-9601-0/minicluster-data/master-1/data/info.pb
I20260806 04:58:04.200089 19751 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 04:58:04.205900 9601 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu as pid 19640
I20260806 04:58:04.206230 19751 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.96.126" port: 40031 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 04:58:04 dist-test-slave-x4qh krb5kdc[18896](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992284, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.96.125@KRBTEST.COM for oryx/127.9.96.126@KRBTEST.COM
Aug 06 04:58:04 dist-test-slave-x4qh krb5kdc[18896](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992281, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.9.96.125@KRBTEST.COM
I20260806 04:58:04.227056 19757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:58:04.209282 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48714 (local address 127.9.96.126:40031)
0806 04:58:04.209807 (+ 525us) server_negotiation.cc:206] Beginning negotiation
0806 04:58:04.209814 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 04:58:04.210461 (+ 647us) server_negotiation.cc:407] Connection header received
0806 04:58:04.211577 (+ 1116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:58:04.211581 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:58:04.211665 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:58:04.211757 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:58:04.213142 (+ 1385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:04.213941 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:04.215720 (+ 1779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:04.216297 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:04.216397 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:58:04.220300 (+ 3903us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:58:04.220308 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:58:04.220313 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:58:04.220354 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:58:04.223551 (+ 3197us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:58:04.224496 (+ 945us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:58:04.224499 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:58:04.224501 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:58:04.224623 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:58:04.225125 (+ 502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:58:04.225133 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:58:04.225136 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:58:04.225552 (+ 416us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:58:04.225702 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0806 04:58:04.226508 (+ 806us) server_negotiation.cc:299] Negotiation successful
0806 04:58:04.226663 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":373,"thread_start_us":243,"threads_started":1}
I20260806 04:58:04.228227 19758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:58:04.209282 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:40031 (local address 127.0.0.1:48714)
0806 04:58:04.210289 (+ 1007us) negotiation.cc:107] Waiting for socket to connect
0806 04:58:04.210348 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0806 04:58:04.211284 (+ 936us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:58:04.211859 (+ 575us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:58:04.211865 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:58:04.212614 (+ 749us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:58:04.212967 (+ 353us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:58:04.212983 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:58:04.214085 (+ 1102us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:58:04.214090 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:58:04.215585 (+ 1495us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:58:04.215597 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:58:04.216532 (+ 935us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:58:04.216539 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:58:04.216748 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:58:04.217504 (+ 756us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:58:04.217524 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:58:04.220102 (+ 2578us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 04:58:04.223691 (+ 3589us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:58:04.223701 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:58:04.223711 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:58:04.224354 (+ 643us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:58:04.224752 (+ 398us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:58:04.224757 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:58:04.224760 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:58:04.224993 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:58:04.225683 (+ 690us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 04:58:04.225691 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 04:58:04.226081 (+ 390us) client_negotiation.cc:769] Sending connection context
0806 04:58:04.226405 (+ 324us) client_negotiation.cc:240] Negotiation successful
0806 04:58:04.226889 (+ 484us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":738,"thread_start_us":510,"threads_started":1}
I20260806 04:58:04.228248 19756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:58:04.208163 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52944 (local address 127.9.96.125:39617)
0806 04:58:04.208971 (+ 808us) server_negotiation.cc:206] Beginning negotiation
0806 04:58:04.208981 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 04:58:04.209017 (+ 36us) server_negotiation.cc:407] Connection header received
0806 04:58:04.211284 (+ 2267us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:58:04.211313 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:58:04.211643 (+ 330us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:58:04.211854 (+ 211us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:58:04.213284 (+ 1430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:04.214712 (+ 1428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:04.216094 (+ 1382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:04.216738 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:04.216879 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:58:04.221662 (+ 4783us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:58:04.221669 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:58:04.221679 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:58:04.221711 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:58:04.224868 (+ 3157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:58:04.225687 (+ 819us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:58:04.225693 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:58:04.225696 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:58:04.225844 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:58:04.226195 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:58:04.226200 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:58:04.226203 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:58:04.226612 (+ 409us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:58:04.226900 (+ 288us) server_negotiation.cc:1035] Waiting for connection context
0806 04:58:04.227605 (+ 705us) server_negotiation.cc:299] Negotiation successful
0806 04:58:04.227829 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":633,"thread_start_us":375,"threads_started":1}
I20260806 04:58:04.228602 19431 heartbeater.cc:503] Master 127.9.96.126:40031 was elected leader, sending a full tablet report...
I20260806 04:58:04.230319 19751 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.96.125" port: 39617 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 04:58:04 dist-test-slave-x4qh krb5kdc[18896](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992284, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.96.125@KRBTEST.COM for oryx/127.9.96.125@KRBTEST.COM
I20260806 04:58:04.241840 19758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:58:04.230959 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.125:39617 (local address 127.0.0.1:52954)
0806 04:58:04.231161 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0806 04:58:04.231183 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0806 04:58:04.231281 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:58:04.231593 (+ 312us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:58:04.231597 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:58:04.231934 (+ 337us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:58:04.232044 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:58:04.232052 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:58:04.232993 (+ 941us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:58:04.232998 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:58:04.233843 (+ 845us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:58:04.233854 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:58:04.234524 (+ 670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:58:04.234529 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:58:04.234781 (+ 252us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:58:04.235201 (+ 420us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:58:04.235216 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:58:04.237083 (+ 1867us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 04:58:04.239927 (+ 2844us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:58:04.239931 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:58:04.239933 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:58:04.240293 (+ 360us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:58:04.240566 (+ 273us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:58:04.240569 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:58:04.240571 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:58:04.240683 (+ 112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:58:04.241133 (+ 450us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 04:58:04.241136 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 04:58:04.241250 (+ 114us) client_negotiation.cc:769] Sending connection context
0806 04:58:04.241350 (+ 100us) client_negotiation.cc:240] Negotiation successful
0806 04:58:04.241593 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":63,"mutex_wait_us":15}
I20260806 04:58:04.242028 19756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:58:04.231182 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52954 (local address 127.9.96.125:39617)
0806 04:58:04.231344 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0806 04:58:04.231351 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 04:58:04.231367 (+ 16us) server_negotiation.cc:407] Connection header received
0806 04:58:04.231428 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:58:04.231432 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:58:04.231486 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:58:04.231577 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:58:04.232167 (+ 590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:04.232856 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:04.233988 (+ 1132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:04.234379 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:04.234516 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:58:04.237265 (+ 2749us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:58:04.237270 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:58:04.237273 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:58:04.237300 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:58:04.239818 (+ 2518us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:58:04.240382 (+ 564us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:58:04.240385 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:58:04.240387 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:58:04.240480 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:58:04.240781 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:58:04.240784 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:58:04.240786 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:58:04.241035 (+ 249us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:58:04.241135 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0806 04:58:04.241431 (+ 296us) server_negotiation.cc:299] Negotiation successful
0806 04:58:04.241693 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":147,"server-negotiator.queue_time_us":52}
I20260806 04:58:04.246554 19751 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 89e9df10c70d4ddaacd2a30fff7a3b2d: Bootstrap starting.
I20260806 04:58:04.249485 19751 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 89e9df10c70d4ddaacd2a30fff7a3b2d: Neither blocks nor log segments found. Creating new log.
I20260806 04:58:04.250489 19751 log.cc:824] T 00000000000000000000000000000000 P 89e9df10c70d4ddaacd2a30fff7a3b2d: Log is configured to *not* fsync() on all Append() calls
I20260806 04:58:04.253036 19751 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 89e9df10c70d4ddaacd2a30fff7a3b2d: No bootstrap required, opened a new log
I20260806 04:58:04.260272 19751 raft_consensus.cc:348] T 00000000000000000000000000000000 P 89e9df10c70d4ddaacd2a30fff7a3b2d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6126c11fed1e4765becce7bd422ce904" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 40031 } } peers { permanent_uuid: "89e9df10c70d4ddaacd2a30fff7a3b2d" member_type: VOTER last_known_addr { host: "127.9.96.125" port: 39617 } }
I20260806 04:58:04.260588 19751 raft_consensus.cc:374] T 00000000000000000000000000000000 P 89e9df10c70d4ddaacd2a30fff7a3b2d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 04:58:04.260691 19751 raft_consensus.cc:729] T 00000000000000000000000000000000 P 89e9df10c70d4ddaacd2a30fff7a3b2d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 89e9df10c70d4ddaacd2a30fff7a3b2d, State: Initialized, Role: FOLLOWER
I20260806 04:58:04.261298 19751 consensus_queue.cc:260] T 00000000000000000000000000000000 P 89e9df10c70d4ddaacd2a30fff7a3b2d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6126c11fed1e4765becce7bd422ce904" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 40031 } } peers { permanent_uuid: "89e9df10c70d4ddaacd2a30fff7a3b2d" member_type: VOTER last_known_addr { host: "127.9.96.125" port: 39617 } }
I20260806 04:58:04.262105 19761 sys_catalog.cc:455] T 00000000000000000000000000000000 P 89e9df10c70d4ddaacd2a30fff7a3b2d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6126c11fed1e4765becce7bd422ce904" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 40031 } } peers { permanent_uuid: "89e9df10c70d4ddaacd2a30fff7a3b2d" member_type: VOTER last_known_addr { host: "127.9.96.125" port: 39617 } } }
I20260806 04:58:04.262385 19761 sys_catalog.cc:458] T 00000000000000000000000000000000 P 89e9df10c70d4ddaacd2a30fff7a3b2d [sys.catalog]: This master's current role is: FOLLOWER
I20260806 04:58:04.263073 19751 sys_catalog.cc:564] T 00000000000000000000000000000000 P 89e9df10c70d4ddaacd2a30fff7a3b2d [sys.catalog]: configured and running, proceeding with master startup.
W20260806 04:58:04.271942 19780 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 89e9df10c70d4ddaacd2a30fff7a3b2d: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 04:58:04.272104 19780 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 04:58:04.274829 19640 master_runner.cc:428] Detected that this master 89e9df10c70d4ddaacd2a30fff7a3b2d is joining an existing cluster
I20260806 04:58:04.274931 19640 master_runner.cc:432] Initiating AddMaster RPC to add 127.9.96.125:39617
I20260806 04:58:04.295295 19756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:58:04.279814 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52966 (local address 127.9.96.125:39617)
0806 04:58:04.279996 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0806 04:58:04.280019 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0806 04:58:04.280375 (+ 356us) server_negotiation.cc:407] Connection header received
0806 04:58:04.280561 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:58:04.280567 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:58:04.280624 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:58:04.280727 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:58:04.281715 (+ 988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:04.282754 (+ 1039us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:04.284774 (+ 2020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:04.285626 (+ 852us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:04.285731 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:58:04.288249 (+ 2518us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:58:04.288257 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:58:04.288264 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:58:04.288306 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:58:04.292165 (+ 3859us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:58:04.293151 (+ 986us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:58:04.293158 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:58:04.293165 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:58:04.293323 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:58:04.293789 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:58:04.293795 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:58:04.293809 (+ 14us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:58:04.294184 (+ 375us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:58:04.294333 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0806 04:58:04.294874 (+ 541us) server_negotiation.cc:299] Negotiation successful
0806 04:58:04.295048 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260806 04:58:04.295445 19757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:58:04.278704 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48718 (local address 127.9.96.126:40031)
0806 04:58:04.278909 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0806 04:58:04.278916 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 04:58:04.279619 (+ 703us) server_negotiation.cc:407] Connection header received
0806 04:58:04.279715 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:58:04.279721 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:58:04.279834 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:58:04.279934 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:58:04.281883 (+ 1949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:04.282987 (+ 1104us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:04.285063 (+ 2076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:04.285702 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:04.285812 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:58:04.288473 (+ 2661us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:58:04.288480 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:58:04.288485 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:58:04.288525 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:58:04.292135 (+ 3610us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:58:04.292982 (+ 847us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:58:04.292990 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:58:04.292993 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:58:04.293169 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:58:04.293670 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:58:04.293676 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:58:04.293679 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:58:04.294047 (+ 368us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:58:04.294183 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0806 04:58:04.294819 (+ 636us) server_negotiation.cc:299] Negotiation successful
0806 04:58:04.295071 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260806 04:58:04.296398 19793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:58:04.279606 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.125:39617 (local address 127.0.0.1:52966)
0806 04:58:04.280280 (+ 674us) negotiation.cc:107] Waiting for socket to connect
0806 04:58:04.280293 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 04:58:04.280431 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:58:04.280739 (+ 308us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:58:04.280743 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:58:04.281315 (+ 572us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:58:04.281580 (+ 265us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:58:04.281587 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:58:04.282986 (+ 1399us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:58:04.282991 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:58:04.284623 (+ 1632us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:58:04.284644 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:58:04.286028 (+ 1384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:58:04.286035 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:58:04.286240 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:58:04.286812 (+ 572us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:58:04.286838 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:58:04.288075 (+ 1237us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 04:58:04.292305 (+ 4230us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:58:04.292315 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:58:04.292322 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:58:04.293011 (+ 689us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:58:04.293482 (+ 471us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:58:04.293489 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:58:04.293494 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:58:04.293675 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:58:04.294429 (+ 754us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 04:58:04.294436 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 04:58:04.294609 (+ 173us) client_negotiation.cc:769] Sending connection context
0806 04:58:04.295508 (+ 899us) client_negotiation.cc:240] Negotiation successful
0806 04:58:04.295818 (+ 310us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":582,"mutex_wait_us":1,"spinlock_wait_cycles":27264,"thread_start_us":413,"threads_started":1}
I20260806 04:58:04.296504 19792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:58:04.278681 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:40031 (local address 127.0.0.1:48718)
0806 04:58:04.279444 (+ 763us) negotiation.cc:107] Waiting for socket to connect
0806 04:58:04.279466 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0806 04:58:04.279598 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:58:04.280131 (+ 533us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:58:04.280137 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:58:04.280911 (+ 774us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:58:04.281415 (+ 504us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:58:04.281428 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:58:04.283159 (+ 1731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:58:04.283168 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:58:04.284895 (+ 1727us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:58:04.284915 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:58:04.286127 (+ 1212us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:58:04.286134 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:58:04.286297 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:58:04.286987 (+ 690us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:58:04.287009 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:58:04.288250 (+ 1241us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 04:58:04.292304 (+ 4054us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:58:04.292316 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:58:04.292320 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:58:04.292847 (+ 527us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:58:04.293336 (+ 489us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:58:04.293340 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:58:04.293343 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:58:04.293531 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:58:04.294277 (+ 746us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 04:58:04.294281 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 04:58:04.294504 (+ 223us) client_negotiation.cc:769] Sending connection context
0806 04:58:04.295615 (+ 1111us) client_negotiation.cc:240] Negotiation successful
0806 04:58:04.295980 (+ 365us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":565,"mutex_wait_us":70,"thread_start_us":419,"threads_started":1}
WARNING: no policy specified for oryx/127.9.96.124@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.9.96.124@KRBTEST.COM" created.
Aug 06 04:58:04 dist-test-slave-x4qh krb5kdc[18896](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992282, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.96.126@KRBTEST.COM for oryx/127.9.96.125@KRBTEST.COM
I20260806 04:58:04.322546 19756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:58:04.303842 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52982 (local address 127.9.96.125:39617)
0806 04:58:04.304002 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0806 04:58:04.304008 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 04:58:04.304278 (+ 270us) server_negotiation.cc:407] Connection header received
0806 04:58:04.304676 (+ 398us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:58:04.304684 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:58:04.304789 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:58:04.304888 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:58:04.306705 (+ 1817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:04.307885 (+ 1180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:04.310021 (+ 2136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:04.310794 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:04.310941 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:58:04.315702 (+ 4761us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:58:04.315713 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:58:04.315717 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:58:04.315755 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:58:04.319325 (+ 3570us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:58:04.320295 (+ 970us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:58:04.320301 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:58:04.320304 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:58:04.320467 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:58:04.320900 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:58:04.320905 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:58:04.320909 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:58:04.321254 (+ 345us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:58:04.321407 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0806 04:58:04.321961 (+ 554us) server_negotiation.cc:299] Negotiation successful
0806 04:58:04.322168 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260806 04:58:04.322597 19796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:58:04.303429 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.125:39617 (local address 127.0.0.1:52982)
0806 04:58:04.304146 (+ 717us) negotiation.cc:107] Waiting for socket to connect
0806 04:58:04.304186 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0806 04:58:04.304433 (+ 247us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:58:04.305201 (+ 768us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:58:04.305218 (+ 17us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:58:04.306130 (+ 912us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:58:04.306495 (+ 365us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:58:04.306510 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:58:04.308057 (+ 1547us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:58:04.308065 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:58:04.309840 (+ 1775us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:58:04.309868 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:58:04.311157 (+ 1289us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:58:04.311164 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:58:04.311417 (+ 253us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:58:04.312335 (+ 918us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:58:04.312360 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:58:04.315517 (+ 3157us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 04:58:04.319528 (+ 4011us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:58:04.319544 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:58:04.319554 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:58:04.320168 (+ 614us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:58:04.320592 (+ 424us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:58:04.320599 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:58:04.320605 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:58:04.320780 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:58:04.321399 (+ 619us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 04:58:04.321409 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 04:58:04.321616 (+ 207us) client_negotiation.cc:769] Sending connection context
0806 04:58:04.321855 (+ 239us) client_negotiation.cc:240] Negotiation successful
0806 04:58:04.322169 (+ 314us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":454,"thread_start_us":579,"threads_started":1}
I20260806 04:58:04.324707 18960 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.9.96.125:39617: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "89e9df10c70d4ddaacd2a30fff7a3b2d"
member_type: NON_VOTER
last_known_addr {
host: "127.9.96.125"
port: 39617
}
attrs {
promote: true
}
}
dest_uuid: "6126c11fed1e4765becce7bd422ce904"
cas_config_opid_index: -1
I20260806 04:58:04.325911 18960 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6126c11fed1e4765becce7bd422ce904 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "6126c11fed1e4765becce7bd422ce904" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 40031 } } peers { permanent_uuid: "89e9df10c70d4ddaacd2a30fff7a3b2d" member_type: NON_VOTER last_known_addr { host: "127.9.96.125" port: 39617 } attrs { promote: true } }
I20260806 04:58:04.334159 19711 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 89e9df10c70d4ddaacd2a30fff7a3b2d [term 0 FOLLOWER]: Advancing to term 1
I20260806 04:58:04.339521 19711 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 89e9df10c70d4ddaacd2a30fff7a3b2d [term 1 FOLLOWER]: Refusing update from remote peer 6126c11fed1e4765becce7bd422ce904: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260806 04:58:04.340674 19801 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 6126c11fed1e4765becce7bd422ce904 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 89e9df10c70d4ddaacd2a30fff7a3b2d (127.9.96.125) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "6126c11fed1e4765becce7bd422ce904" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 40031 } } peers { permanent_uuid: "89e9df10c70d4ddaacd2a30fff7a3b2d" member_type: NON_VOTER last_known_addr { host: "127.9.96.125" port: 39617 } attrs { promote: true } } }
I20260806 04:58:04.343890 19799 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 6126c11fed1e4765becce7bd422ce904 [LEADER]: Connected to new peer: Peer: permanent_uuid: "89e9df10c70d4ddaacd2a30fff7a3b2d" member_type: NON_VOTER last_known_addr { host: "127.9.96.125" port: 39617 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.9.96.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785992227050689-9601-0/krb5kdc/oryx_127.9.96.124.keytab.
Entry for principal oryx/127.9.96.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785992227050689-9601-0/krb5kdc/oryx_127.9.96.124.keytab.
I20260806 04:58:04.351985 19801 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.9.96.125:39617
I20260806 04:58:04.352671 9601 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
/tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785992227050689-9601-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785992227050689-9601-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785992227050689-9601-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785992227050689-9601-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.96.124:44477
--webserver_interface=127.9.96.124
--webserver_port=0
--keytab_file=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785992227050689-9601-0/krb5kdc/oryx_127.9.96.124.keytab
--principal=oryx/127.9.96.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.96.84:34171
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.96.126:40031,127.9.96.125:39617,127.9.96.124:44477
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785992227050689-9601-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785992227050689-9601-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785992227050689-9601-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 04:58:04.352861 19640 master.cc:537] Master@127.9.96.125:39617 shutting down...
I20260806 04:58:04.352666 19802 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6126c11fed1e4765becce7bd422ce904 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "6126c11fed1e4765becce7bd422ce904" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "6126c11fed1e4765becce7bd422ce904" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 40031 } } peers { permanent_uuid: "89e9df10c70d4ddaacd2a30fff7a3b2d" member_type: NON_VOTER last_known_addr { host: "127.9.96.125" port: 39617 } attrs { promote: true } } }
I20260806 04:58:04.352986 19802 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6126c11fed1e4765becce7bd422ce904 [sys.catalog]: This master's current role is: LEADER
I20260806 04:58:04.381892 19711 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 89e9df10c70d4ddaacd2a30fff7a3b2d [term 1 LEARNER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 89e9df10c70d4ddaacd2a30fff7a3b2d (127.9.96.125) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "6126c11fed1e4765becce7bd422ce904" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 40031 } } peers { permanent_uuid: "89e9df10c70d4ddaacd2a30fff7a3b2d" member_type: NON_VOTER last_known_addr { host: "127.9.96.125" port: 39617 } attrs { promote: true } } }
I20260806 04:58:04.394919 19813 sys_catalog.cc:455] T 00000000000000000000000000000000 P 89e9df10c70d4ddaacd2a30fff7a3b2d [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "6126c11fed1e4765becce7bd422ce904" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "6126c11fed1e4765becce7bd422ce904" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 40031 } } peers { permanent_uuid: "89e9df10c70d4ddaacd2a30fff7a3b2d" member_type: NON_VOTER last_known_addr { host: "127.9.96.125" port: 39617 } attrs { promote: true } } }
I20260806 04:58:04.395416 19813 sys_catalog.cc:458] T 00000000000000000000000000000000 P 89e9df10c70d4ddaacd2a30fff7a3b2d [sys.catalog]: This master's current role is: LEARNER
I20260806 04:58:04.395623 19761 sys_catalog.cc:455] T 00000000000000000000000000000000 P 89e9df10c70d4ddaacd2a30fff7a3b2d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6126c11fed1e4765becce7bd422ce904. Latest consensus state: current_term: 1 leader_uuid: "6126c11fed1e4765becce7bd422ce904" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "6126c11fed1e4765becce7bd422ce904" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 40031 } } peers { permanent_uuid: "89e9df10c70d4ddaacd2a30fff7a3b2d" member_type: NON_VOTER last_known_addr { host: "127.9.96.125" port: 39617 } attrs { promote: true } } }
I20260806 04:58:04.395839 19761 sys_catalog.cc:458] T 00000000000000000000000000000000 P 89e9df10c70d4ddaacd2a30fff7a3b2d [sys.catalog]: This master's current role is: LEARNER
I20260806 04:58:04.395961 19808 sys_catalog.cc:455] T 00000000000000000000000000000000 P 89e9df10c70d4ddaacd2a30fff7a3b2d [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "6126c11fed1e4765becce7bd422ce904" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "6126c11fed1e4765becce7bd422ce904" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 40031 } } peers { permanent_uuid: "89e9df10c70d4ddaacd2a30fff7a3b2d" member_type: NON_VOTER last_known_addr { host: "127.9.96.125" port: 39617 } attrs { promote: true } } }
I20260806 04:58:04.396292 19808 sys_catalog.cc:458] T 00000000000000000000000000000000 P 89e9df10c70d4ddaacd2a30fff7a3b2d [sys.catalog]: This master's current role is: LEARNER
I20260806 04:58:04.398439 19640 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 89e9df10c70d4ddaacd2a30fff7a3b2d [term 1 LEARNER]: Raft consensus shutting down.
I20260806 04:58:04.398895 19640 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 89e9df10c70d4ddaacd2a30fff7a3b2d [term 1 LEARNER]: Raft consensus is shut down!
I20260806 04:58:04.399068 19640 tablet_replica.cc:330] T 00000000000000000000000000000000 P 89e9df10c70d4ddaacd2a30fff7a3b2d: stopping tablet replica
I20260806 04:58:04.406188 19640 master.cc:552] Master@127.9.96.125:39617 shutdown complete.
I20260806 04:58:04.406316 19640 master_runner.cc:305] Clearing existing system tablet
I20260806 04:58:04.406787 19640 fs_manager.cc:359] Metadata directory not provided
I20260806 04:58:04.406898 19640 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 04:58:04.415534 19640 fs_manager.cc:709] Time spent opening directory manager: real 0.008s user 0.007s sys 0.001s
I20260806 04:58:04.416966 19815 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 04:58:04.417300 19640 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785992227050689-9601-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 04:58:04.417644 19640 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 04:58:04.417734 19640 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785992227050689-9601-0/minicluster-data/master-1/data,/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785992227050689-9601-0/minicluster-data/master-1/wal
uuid: "89e9df10c70d4ddaacd2a30fff7a3b2d"
format_stamp: "Formatted at 2026-08-06 04:58:04 on dist-test-slave-x4qh"
server_key: "95dff7d38bf45a98e8a13c1e20ce4a30"
server_key_iv: "44aa72342ba02f364935e50cb4c16f16"
server_key_version: "encryptionkey@0"
I20260806 04:58:04.421092 19640 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 89e9df10c70d4ddaacd2a30fff7a3b2d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 04:58:04.425637 19640 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 89e9df10c70d4ddaacd2a30fff7a3b2d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260806 04:58:04.425822 19640 log.cc:1196] T 00000000000000000000000000000000 P 89e9df10c70d4ddaacd2a30fff7a3b2d: Deleting WAL directory at /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785992227050689-9601-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260806 04:58:04.426241 19640 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 89e9df10c70d4ddaacd2a30fff7a3b2d: Deleting consensus metadata
I20260806 04:58:04.426496 19640 master_runner.cc:315] Copying system tablet from 127.9.96.126:40031
I20260806 04:58:04.429399 19640 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 89e9df10c70d4ddaacd2a30fff7a3b2d: tablet copy: Beginning tablet copy session from remote peer at address 127.9.96.126:40031
I20260806 04:58:04.443331 19757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:58:04.430286 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48724 (local address 127.9.96.126:40031)
0806 04:58:04.430508 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0806 04:58:04.430517 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 04:58:04.431920 (+ 1403us) server_negotiation.cc:407] Connection header received
0806 04:58:04.432120 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:58:04.432126 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:58:04.432215 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:58:04.432329 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:58:04.433162 (+ 833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:04.434108 (+ 946us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:04.435264 (+ 1156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:04.435731 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:04.435861 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:58:04.437765 (+ 1904us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:58:04.437771 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:58:04.437775 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:58:04.437813 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:58:04.440746 (+ 2933us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:58:04.441349 (+ 603us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:58:04.441355 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:58:04.441359 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:58:04.441523 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:58:04.441928 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:58:04.441934 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:58:04.441938 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:58:04.442194 (+ 256us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:58:04.442360 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0806 04:58:04.442741 (+ 381us) server_negotiation.cc:299] Negotiation successful
0806 04:58:04.442942 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260806 04:58:04.443405 19822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:58:04.430254 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:40031 (local address 127.0.0.1:48724)
0806 04:58:04.431805 (+ 1551us) negotiation.cc:107] Waiting for socket to connect
0806 04:58:04.431826 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0806 04:58:04.431986 (+ 160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:58:04.432348 (+ 362us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:58:04.432353 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:58:04.432789 (+ 436us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:58:04.433013 (+ 224us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:58:04.433023 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:58:04.434253 (+ 1230us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:58:04.434258 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:58:04.435119 (+ 861us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:58:04.435130 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:58:04.435875 (+ 745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:58:04.435880 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:58:04.436054 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:58:04.436674 (+ 620us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:58:04.436691 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:58:04.437598 (+ 907us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 04:58:04.440884 (+ 3286us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:58:04.440889 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:58:04.440893 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:58:04.441229 (+ 336us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:58:04.441662 (+ 433us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:58:04.441667 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:58:04.441671 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:58:04.441820 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:58:04.442317 (+ 497us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 04:58:04.442322 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 04:58:04.442504 (+ 182us) client_negotiation.cc:769] Sending connection context
0806 04:58:04.442639 (+ 135us) client_negotiation.cc:240] Negotiation successful
0806 04:58:04.442953 (+ 314us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1429,"thread_start_us":1294,"threads_started":1}
I20260806 04:58:04.444415 18990 tablet_copy_service.cc:140] P 6126c11fed1e4765becce7bd422ce904: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 89e9df10c70d4ddaacd2a30fff7a3b2d ({username='oryx', principal='oryx/127.9.96.125@KRBTEST.COM'} at 127.0.0.1:48724)
I20260806 04:58:04.444821 18990 tablet_copy_service.cc:161] P 6126c11fed1e4765becce7bd422ce904: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 89e9df10c70d4ddaacd2a30fff7a3b2d at {username='oryx', principal='oryx/127.9.96.125@KRBTEST.COM'} at 127.0.0.1:48724: session id = 89e9df10c70d4ddaacd2a30fff7a3b2d-00000000000000000000000000000000
I20260806 04:58:04.448282 18990 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 6126c11fed1e4765becce7bd422ce904: Tablet Copy: opened 0 blocks and 1 log segments
I20260806 04:58:04.450318 19640 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260806 04:58:04.455319 19640 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 89e9df10c70d4ddaacd2a30fff7a3b2d: tablet copy: Starting download of 0 data blocks...
I20260806 04:58:04.455766 19640 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 89e9df10c70d4ddaacd2a30fff7a3b2d: tablet copy: Starting download of 1 WAL segments...
I20260806 04:58:04.458982 19640 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 89e9df10c70d4ddaacd2a30fff7a3b2d: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260806 04:58:04.462597 18990 tablet_copy_service.cc:342] P 6126c11fed1e4765becce7bd422ce904: Request end of tablet copy session 89e9df10c70d4ddaacd2a30fff7a3b2d-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.9.96.125@KRBTEST.COM'} at 127.0.0.1:48724
I20260806 04:58:04.462838 18990 tablet_copy_service.cc:433] P 6126c11fed1e4765becce7bd422ce904: ending tablet copy session 89e9df10c70d4ddaacd2a30fff7a3b2d-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 89e9df10c70d4ddaacd2a30fff7a3b2d
I20260806 04:58:04.474288 19640 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 04:58:04.477237 19823 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:58:04.483228 19824 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:58:04.488201 19827 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:58:04.488560 19640 server_base.cc:1034] running on GCE node
I20260806 04:58:04.488970 19640 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 04:58:04.489699 19640 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 04:58:04.490931 19640 hybrid_clock.cc:648] HybridClock initialized: now 1785992284490859 us; error 41 us; skew 500 ppm
I20260806 04:58:04.491184 19640 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 04:58:04 dist-test-slave-x4qh krb5kdc[18896](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992284, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.96.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 04:58:04.493296 19640 init.cc:377] Logged in from keytab as oryx/127.9.96.125@KRBTEST.COM (short username oryx)
I20260806 04:58:04.498343 19640 webserver.cc:460] Webserver started at http://127.9.96.125:43635/ using document root <none> and password file <none>
I20260806 04:58:04.498703 19640 fs_manager.cc:359] Metadata directory not provided
I20260806 04:58:04.498821 19640 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 04:58:04.501485 19640 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.001s sys 0.001s
I20260806 04:58:04.503286 19840 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 04:58:04.503757 19640 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 04:58:04.503890 19640 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785992227050689-9601-0/minicluster-data/master-1/data,/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785992227050689-9601-0/minicluster-data/master-1/wal
uuid: "89e9df10c70d4ddaacd2a30fff7a3b2d"
format_stamp: "Formatted at 2026-08-06 04:58:04 on dist-test-slave-x4qh"
server_key: "95dff7d38bf45a98e8a13c1e20ce4a30"
server_key_iv: "44aa72342ba02f364935e50cb4c16f16"
server_key_version: "encryptionkey@0"
I20260806 04:58:04.504071 19640 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785992227050689-9601-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785992227050689-9601-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785992227050689-9601-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 04:58:04.517050 19640 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 04:58:04.520646 19640 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 04:58:04.543713 19640 rpc_server.cc:307] RPC server started. Bound to: 127.9.96.125:39617
I20260806 04:58:04.543754 19923 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.96.125:39617 every 8 connection(s)
I20260806 04:58:04.544641 19640 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785992227050689-9601-0/minicluster-data/master-1/data/info.pb
I20260806 04:58:04.551661 19925 sys_catalog.cc:263] Verifying existing consensus state
I20260806 04:58:04.554138 19925 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 89e9df10c70d4ddaacd2a30fff7a3b2d: Bootstrap starting.
I20260806 04:58:04.565117 19925 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 89e9df10c70d4ddaacd2a30fff7a3b2d: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 04:58:04.565712 19925 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 89e9df10c70d4ddaacd2a30fff7a3b2d: Bootstrap complete.
I20260806 04:58:04.566630 19925 raft_consensus.cc:348] T 00000000000000000000000000000000 P 89e9df10c70d4ddaacd2a30fff7a3b2d [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "6126c11fed1e4765becce7bd422ce904" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 40031 } } peers { permanent_uuid: "89e9df10c70d4ddaacd2a30fff7a3b2d" member_type: NON_VOTER last_known_addr { host: "127.9.96.125" port: 39617 } attrs { promote: true } }
I20260806 04:58:04.566874 19925 raft_consensus.cc:729] T 00000000000000000000000000000000 P 89e9df10c70d4ddaacd2a30fff7a3b2d [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 89e9df10c70d4ddaacd2a30fff7a3b2d, State: Initialized, Role: LEARNER
I20260806 04:58:04.567124 19925 consensus_queue.cc:260] T 00000000000000000000000000000000 P 89e9df10c70d4ddaacd2a30fff7a3b2d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "6126c11fed1e4765becce7bd422ce904" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 40031 } } peers { permanent_uuid: "89e9df10c70d4ddaacd2a30fff7a3b2d" member_type: NON_VOTER last_known_addr { host: "127.9.96.125" port: 39617 } attrs { promote: true } }
I20260806 04:58:04.567934 19929 sys_catalog.cc:455] T 00000000000000000000000000000000 P 89e9df10c70d4ddaacd2a30fff7a3b2d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "6126c11fed1e4765becce7bd422ce904" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 40031 } } peers { permanent_uuid: "89e9df10c70d4ddaacd2a30fff7a3b2d" member_type: NON_VOTER last_known_addr { host: "127.9.96.125" port: 39617 } attrs { promote: true } } }
I20260806 04:58:04.568286 19925 sys_catalog.cc:564] T 00000000000000000000000000000000 P 89e9df10c70d4ddaacd2a30fff7a3b2d [sys.catalog]: configured and running, proceeding with master startup.
I20260806 04:58:04.568396 19929 sys_catalog.cc:458] T 00000000000000000000000000000000 P 89e9df10c70d4ddaacd2a30fff7a3b2d [sys.catalog]: This master's current role is: LEARNER
I20260806 04:58:04.577790 19944 catalog_manager.cc:1259] Loaded cluster ID: b1cd49f4c4814e84b0f51af05de8eeee
I20260806 04:58:04.577903 19944 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 89e9df10c70d4ddaacd2a30fff7a3b2d: loading cluster ID for follower catalog manager: success
I20260806 04:58:04.580384 19944 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 89e9df10c70d4ddaacd2a30fff7a3b2d: acquiring CA information for follower catalog manager: success
I20260806 04:58:04.582175 19944 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 89e9df10c70d4ddaacd2a30fff7a3b2d: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 04:58:04.713817 19633 heartbeater.cc:503] Master 127.9.96.126:40031 was elected leader, sending a full tablet report...
W20260806 04:58:04.816489 19806 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 04:58:04.816849 19806 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 04:58:04.816985 19806 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 04:58:04.825493 19806 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 04:58:04.825596 19806 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 04:58:04.825690 19806 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 04:58:04.825766 19806 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 04:58:04.825860 19806 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 04:58:04.835631 19806 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.96.84:34171
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785992227050689-9601-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785992227050689-9601-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.9.96.126:40031,127.9.96.125:39617,127.9.96.124:44477
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785992227050689-9601-0/krb5kdc/oryx_127.9.96.124.keytab
--principal=oryx/127.9.96.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.96.124:44477
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785992227050689-9601-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.9.96.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785992227050689-9601-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 03:56:12 UTC on ccycloud.quasar-guijbm.root.comops.site
build id 4569618
ASAN enabled
I20260806 04:58:04.836925 19806 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 04:58:04.838430 19806 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 04:58:04.850143 19950 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:58:04.850173 19951 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:58:04.851265 19953 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:58:04.851341 19806 server_base.cc:1034] running on GCE node
I20260806 04:58:04.852142 19806 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 04:58:04.853458 19806 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 04:58:04.854728 19806 hybrid_clock.cc:648] HybridClock initialized: now 1785992284854663 us; error 54 us; skew 500 ppm
I20260806 04:58:04.855069 19806 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 04:58:04 dist-test-slave-x4qh krb5kdc[18896](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992284, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.96.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 04:58:04.861912 19806 init.cc:377] Logged in from keytab as oryx/127.9.96.124@KRBTEST.COM (short username oryx)
I20260806 04:58:04.863467 19806 webserver.cc:460] Webserver started at http://127.9.96.124:37331/ using document root <none> and password file <none>
I20260806 04:58:04.864110 19806 fs_manager.cc:359] Metadata directory not provided
I20260806 04:58:04.864277 19806 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 04:58:04.864578 19806 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 04:58:04.867528 19806 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785992227050689-9601-0/minicluster-data/master-2/data/instance:
uuid: "50f00ca72a174ac4a6a7f39da3065462"
format_stamp: "Formatted at 2026-08-06 04:58:04 on dist-test-slave-x4qh"
server_key: "74e377d6477d8894f812546c5013ec32"
server_key_iv: "ef898b742ca2a6e0a0d35d36d52dee79"
server_key_version: "encryptionkey@0"
I20260806 04:58:04.868352 19806 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785992227050689-9601-0/minicluster-data/master-2/wal/instance:
uuid: "50f00ca72a174ac4a6a7f39da3065462"
format_stamp: "Formatted at 2026-08-06 04:58:04 on dist-test-slave-x4qh"
server_key: "74e377d6477d8894f812546c5013ec32"
server_key_iv: "ef898b742ca2a6e0a0d35d36d52dee79"
server_key_version: "encryptionkey@0"
I20260806 04:58:04.873332 19806 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260806 04:58:04.876690 19966 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 04:58:04.878211 19806 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 04:58:04.878425 19806 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785992227050689-9601-0/minicluster-data/master-2/data,/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785992227050689-9601-0/minicluster-data/master-2/wal
uuid: "50f00ca72a174ac4a6a7f39da3065462"
format_stamp: "Formatted at 2026-08-06 04:58:04 on dist-test-slave-x4qh"
server_key: "74e377d6477d8894f812546c5013ec32"
server_key_iv: "ef898b742ca2a6e0a0d35d36d52dee79"
server_key_version: "encryptionkey@0"
I20260806 04:58:04.878628 19806 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785992227050689-9601-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785992227050689-9601-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785992227050689-9601-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 04:58:04.895145 19802 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6126c11fed1e4765becce7bd422ce904 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "6126c11fed1e4765becce7bd422ce904" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "6126c11fed1e4765becce7bd422ce904" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 40031 } } peers { permanent_uuid: "89e9df10c70d4ddaacd2a30fff7a3b2d" member_type: NON_VOTER last_known_addr { host: "127.9.96.125" port: 39617 } attrs { promote: true } } }
I20260806 04:58:04.895552 19802 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6126c11fed1e4765becce7bd422ce904 [sys.catalog]: This master's current role is: LEADER
I20260806 04:58:04.899672 19806 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 04:58:04.906630 19806 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 04:58:04.906992 19806 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 04:58:04.920202 19978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:58:04.895256 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.125:39617 (local address 127.0.0.1:52988)
0806 04:58:04.913045 (+ 17789us) negotiation.cc:107] Waiting for socket to connect
0806 04:58:04.913068 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0806 04:58:04.913238 (+ 170us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:58:04.913684 (+ 446us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:58:04.913693 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:58:04.913725 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 04:58:04.913952 (+ 227us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:58:04.913966 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:58:04.915300 (+ 1334us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:58:04.915306 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:58:04.917613 (+ 2307us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:58:04.917638 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:58:04.919110 (+ 1472us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:58:04.919118 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:58:04.919365 (+ 247us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:58:04.919473 (+ 108us) client_negotiation.cc:769] Sending connection context
0806 04:58:04.919609 (+ 136us) client_negotiation.cc:240] Negotiation successful
0806 04:58:04.919776 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":17661,"thread_start_us":17548,"threads_started":1}
I20260806 04:58:04.920229 19981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:58:04.896185 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52988 (local address 127.9.96.125:39617)
0806 04:58:04.912235 (+ 16050us) server_negotiation.cc:206] Beginning negotiation
0806 04:58:04.912245 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 04:58:04.913171 (+ 926us) server_negotiation.cc:407] Connection header received
0806 04:58:04.913414 (+ 243us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:58:04.913418 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:58:04.913492 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:58:04.913587 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 04:58:04.914122 (+ 535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:04.915128 (+ 1006us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:04.917821 (+ 2693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:04.918916 (+ 1095us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:04.919066 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:58:04.919358 (+ 292us) server_negotiation.cc:1035] Waiting for connection context
0806 04:58:04.919645 (+ 287us) server_negotiation.cc:299] Negotiation successful
0806 04:58:04.919800 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15840,"thread_start_us":15693,"threads_started":1}
I20260806 04:58:04.921448 19929 sys_catalog.cc:455] T 00000000000000000000000000000000 P 89e9df10c70d4ddaacd2a30fff7a3b2d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6126c11fed1e4765becce7bd422ce904. Latest consensus state: current_term: 1 leader_uuid: "6126c11fed1e4765becce7bd422ce904" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "6126c11fed1e4765becce7bd422ce904" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 40031 } } peers { permanent_uuid: "89e9df10c70d4ddaacd2a30fff7a3b2d" member_type: NON_VOTER last_known_addr { host: "127.9.96.125" port: 39617 } attrs { promote: true } } }
I20260806 04:58:04.921769 19929 sys_catalog.cc:458] T 00000000000000000000000000000000 P 89e9df10c70d4ddaacd2a30fff7a3b2d [sys.catalog]: This master's current role is: LEARNER
I20260806 04:58:04.922528 19975 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 6126c11fed1e4765becce7bd422ce904: attempting to promote NON_VOTER 89e9df10c70d4ddaacd2a30fff7a3b2d to VOTER
I20260806 04:58:04.923570 19975 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6126c11fed1e4765becce7bd422ce904 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "6126c11fed1e4765becce7bd422ce904" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 40031 } } peers { permanent_uuid: "89e9df10c70d4ddaacd2a30fff7a3b2d" member_type: VOTER last_known_addr { host: "127.9.96.125" port: 39617 } attrs { promote: false } }
I20260806 04:58:04.925576 19885 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 89e9df10c70d4ddaacd2a30fff7a3b2d [term 1 LEARNER]: Refusing update from remote peer 6126c11fed1e4765becce7bd422ce904: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260806 04:58:04.926208 19802 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 6126c11fed1e4765becce7bd422ce904 [LEADER]: Connected to new peer: Peer: permanent_uuid: "89e9df10c70d4ddaacd2a30fff7a3b2d" member_type: VOTER last_known_addr { host: "127.9.96.125" port: 39617 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260806 04:58:04.929298 19975 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 6126c11fed1e4765becce7bd422ce904 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 89e9df10c70d4ddaacd2a30fff7a3b2d (127.9.96.125) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "6126c11fed1e4765becce7bd422ce904" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 40031 } } peers { permanent_uuid: "89e9df10c70d4ddaacd2a30fff7a3b2d" member_type: VOTER last_known_addr { host: "127.9.96.125" port: 39617 } attrs { promote: false } } }
I20260806 04:58:04.930008 19885 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 89e9df10c70d4ddaacd2a30fff7a3b2d [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 89e9df10c70d4ddaacd2a30fff7a3b2d (127.9.96.125) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "6126c11fed1e4765becce7bd422ce904" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 40031 } } peers { permanent_uuid: "89e9df10c70d4ddaacd2a30fff7a3b2d" member_type: VOTER last_known_addr { host: "127.9.96.125" port: 39617 } attrs { promote: false } } }
I20260806 04:58:04.933418 19806 rpc_server.cc:307] RPC server started. Bound to: 127.9.96.124:44477
I20260806 04:58:04.933460 20063 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.96.124:44477 every 8 connection(s)
I20260806 04:58:04.933705 19974 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6126c11fed1e4765becce7bd422ce904 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "6126c11fed1e4765becce7bd422ce904" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "6126c11fed1e4765becce7bd422ce904" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 40031 } } peers { permanent_uuid: "89e9df10c70d4ddaacd2a30fff7a3b2d" member_type: VOTER last_known_addr { host: "127.9.96.125" port: 39617 } attrs { promote: false } } }
I20260806 04:58:04.934008 19974 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6126c11fed1e4765becce7bd422ce904 [sys.catalog]: This master's current role is: LEADER
I20260806 04:58:04.934098 19929 sys_catalog.cc:455] T 00000000000000000000000000000000 P 89e9df10c70d4ddaacd2a30fff7a3b2d [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "6126c11fed1e4765becce7bd422ce904" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "6126c11fed1e4765becce7bd422ce904" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 40031 } } peers { permanent_uuid: "89e9df10c70d4ddaacd2a30fff7a3b2d" member_type: VOTER last_known_addr { host: "127.9.96.125" port: 39617 } attrs { promote: false } } }
I20260806 04:58:04.934315 19929 sys_catalog.cc:458] T 00000000000000000000000000000000 P 89e9df10c70d4ddaacd2a30fff7a3b2d [sys.catalog]: This master's current role is: FOLLOWER
I20260806 04:58:04.934319 19974 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6126c11fed1e4765becce7bd422ce904 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "6126c11fed1e4765becce7bd422ce904" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "6126c11fed1e4765becce7bd422ce904" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 40031 } } peers { permanent_uuid: "89e9df10c70d4ddaacd2a30fff7a3b2d" member_type: VOTER last_known_addr { host: "127.9.96.125" port: 39617 } attrs { promote: false } } }
I20260806 04:58:04.934517 19974 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6126c11fed1e4765becce7bd422ce904 [sys.catalog]: This master's current role is: LEADER
I20260806 04:58:04.935040 19806 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785992227050689-9601-0/minicluster-data/master-2/data/info.pb
I20260806 04:58:04.936305 9601 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu as pid 19806
I20260806 04:58:04.940280 20064 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 04:58:04.949687 20064 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.96.126" port: 40031 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 04:58:04 dist-test-slave-x4qh krb5kdc[18896](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992281, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.9.96.124@KRBTEST.COM
Aug 06 04:58:04 dist-test-slave-x4qh krb5kdc[18896](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992284, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.96.124@KRBTEST.COM for oryx/127.9.96.126@KRBTEST.COM
I20260806 04:58:04.964782 20069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:58:04.938472 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55138 (local address 127.9.96.124:44477)
0806 04:58:04.939268 (+ 796us) server_negotiation.cc:206] Beginning negotiation
0806 04:58:04.939300 (+ 32us) server_negotiation.cc:399] Waiting for connection header
0806 04:58:04.943881 (+ 4581us) server_negotiation.cc:407] Connection header received
0806 04:58:04.945755 (+ 1874us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:58:04.945776 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:58:04.946191 (+ 415us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:58:04.946560 (+ 369us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:58:04.948034 (+ 1474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:04.949296 (+ 1262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:04.950630 (+ 1334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:04.951619 (+ 989us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:04.951846 (+ 227us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:58:04.957207 (+ 5361us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:58:04.957216 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:58:04.957227 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:58:04.957265 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:58:04.960315 (+ 3050us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:58:04.961119 (+ 804us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:58:04.961125 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:58:04.961130 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:58:04.961284 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:58:04.961668 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:58:04.961673 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:58:04.961676 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:58:04.962326 (+ 650us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:58:04.962645 (+ 319us) server_negotiation.cc:1035] Waiting for connection context
0806 04:58:04.963001 (+ 356us) server_negotiation.cc:299] Negotiation successful
0806 04:58:04.963463 (+ 462us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":577,"thread_start_us":363,"threads_started":1}
I20260806 04:58:04.968827 20072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:58:04.952102 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:40031 (local address 127.0.0.1:48736)
0806 04:58:04.952757 (+ 655us) negotiation.cc:107] Waiting for socket to connect
0806 04:58:04.952805 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0806 04:58:04.952965 (+ 160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:58:04.953380 (+ 415us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:58:04.953386 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:58:04.953968 (+ 582us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:58:04.954201 (+ 233us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:58:04.954209 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:58:04.955250 (+ 1041us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:58:04.955256 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:58:04.956924 (+ 1668us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:58:04.956939 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:58:04.957940 (+ 1001us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:58:04.957948 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:58:04.958173 (+ 225us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:58:04.958815 (+ 642us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:58:04.958861 (+ 46us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:58:04.961305 (+ 2444us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 04:58:04.965260 (+ 3955us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:58:04.965269 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:58:04.965273 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:58:04.965771 (+ 498us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:58:04.967342 (+ 1571us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:58:04.967345 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:58:04.967348 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:58:04.967522 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:58:04.968083 (+ 561us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 04:58:04.968088 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 04:58:04.968217 (+ 129us) client_negotiation.cc:769] Sending connection context
0806 04:58:04.968351 (+ 134us) client_negotiation.cc:240] Negotiation successful
0806 04:58:04.968554 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":438,"thread_start_us":308,"threads_started":1}
I20260806 04:58:04.968961 20074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:58:04.952255 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48736 (local address 127.9.96.126:40031)
0806 04:58:04.952700 (+ 445us) server_negotiation.cc:206] Beginning negotiation
0806 04:58:04.952707 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 04:58:04.952899 (+ 192us) server_negotiation.cc:407] Connection header received
0806 04:58:04.953102 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:58:04.953110 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:58:04.953199 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:58:04.953329 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:58:04.954351 (+ 1022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:04.955108 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:04.957071 (+ 1963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:04.957736 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:04.957895 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:58:04.961496 (+ 3601us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:58:04.961501 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:58:04.961504 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:58:04.961533 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:58:04.965130 (+ 3597us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:58:04.965887 (+ 757us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:58:04.965893 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:58:04.965897 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:58:04.967172 (+ 1275us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:58:04.967631 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:58:04.967635 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:58:04.967638 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:58:04.967979 (+ 341us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:58:04.968091 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0806 04:58:04.968436 (+ 345us) server_negotiation.cc:299] Negotiation successful
0806 04:58:04.968646 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":345,"thread_start_us":251,"threads_started":1}
I20260806 04:58:04.970054 20064 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.96.125" port: 39617 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 04:58:04 dist-test-slave-x4qh krb5kdc[18896](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992284, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.96.124@KRBTEST.COM for oryx/127.9.96.125@KRBTEST.COM
I20260806 04:58:04.981006 20072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:58:04.970518 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.125:39617 (local address 127.0.0.1:52998)
0806 04:58:04.970686 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0806 04:58:04.970706 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0806 04:58:04.970798 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:58:04.971055 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:58:04.971058 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:58:04.971332 (+ 274us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:58:04.971516 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:58:04.971525 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:58:04.972556 (+ 1031us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:58:04.972559 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:58:04.973527 (+ 968us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:58:04.973546 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:58:04.974160 (+ 614us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:58:04.974163 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:58:04.974279 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:58:04.974702 (+ 423us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:58:04.974717 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:58:04.976499 (+ 1782us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 04:58:04.979113 (+ 2614us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:58:04.979116 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:58:04.979119 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:58:04.979535 (+ 416us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:58:04.979845 (+ 310us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:58:04.979848 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:58:04.979850 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:58:04.979978 (+ 128us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:58:04.980411 (+ 433us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 04:58:04.980414 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 04:58:04.980522 (+ 108us) client_negotiation.cc:769] Sending connection context
0806 04:58:04.980617 (+ 95us) client_negotiation.cc:240] Negotiation successful
0806 04:58:04.980797 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":46}
I20260806 04:58:04.981156 19981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:58:04.970640 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52998 (local address 127.9.96.125:39617)
0806 04:58:04.970824 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0806 04:58:04.970829 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 04:58:04.970844 (+ 15us) server_negotiation.cc:407] Connection header received
0806 04:58:04.970891 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:58:04.970894 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:58:04.970947 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:58:04.971027 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:58:04.971641 (+ 614us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:04.972442 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:04.973665 (+ 1223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:04.974045 (+ 380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:04.974134 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:58:04.976660 (+ 2526us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:58:04.976664 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:58:04.976667 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:58:04.976698 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:58:04.979002 (+ 2304us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:58:04.979627 (+ 625us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:58:04.979630 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:58:04.979632 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:58:04.979754 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:58:04.980068 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:58:04.980071 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:58:04.980073 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:58:04.980320 (+ 247us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:58:04.980427 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0806 04:58:04.980711 (+ 284us) server_negotiation.cc:299] Negotiation successful
0806 04:58:04.980900 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260806 04:58:04.981931 20064 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.96.124" port: 44477 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 04:58:04 dist-test-slave-x4qh krb5kdc[18896](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992284, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.96.124@KRBTEST.COM for oryx/127.9.96.124@KRBTEST.COM
I20260806 04:58:04.993605 20072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:58:04.982755 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.124:44477 (local address 127.0.0.1:55154)
0806 04:58:04.983007 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0806 04:58:04.983020 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 04:58:04.983100 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:58:04.983360 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:58:04.983363 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:58:04.983734 (+ 371us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:58:04.983875 (+ 141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:58:04.983883 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:58:04.984746 (+ 863us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:58:04.984749 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:58:04.985691 (+ 942us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:58:04.985702 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:58:04.986325 (+ 623us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:58:04.986330 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:58:04.986484 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:58:04.986922 (+ 438us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:58:04.986938 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:58:04.988809 (+ 1871us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 04:58:04.991305 (+ 2496us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:58:04.991311 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:58:04.991315 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:58:04.991703 (+ 388us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:58:04.992049 (+ 346us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:58:04.992054 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:58:04.992058 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:58:04.992202 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:58:04.992688 (+ 486us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 04:58:04.992692 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 04:58:04.992830 (+ 138us) client_negotiation.cc:769] Sending connection context
0806 04:58:04.992973 (+ 143us) client_negotiation.cc:240] Negotiation successful
0806 04:58:04.993236 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":169}
I20260806 04:58:04.993721 20069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:58:04.982751 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55154 (local address 127.9.96.124:44477)
0806 04:58:04.982907 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0806 04:58:04.982911 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 04:58:04.983082 (+ 171us) server_negotiation.cc:407] Connection header received
0806 04:58:04.983184 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:58:04.983188 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:58:04.983252 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:58:04.983325 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:58:04.983988 (+ 663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:04.984628 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:04.985804 (+ 1176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:04.986186 (+ 382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:04.986281 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:58:04.988984 (+ 2703us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:58:04.988988 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:58:04.988991 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:58:04.989016 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:58:04.991176 (+ 2160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:58:04.991812 (+ 636us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:58:04.991820 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:58:04.991823 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:58:04.991930 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:58:04.992309 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:58:04.992314 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:58:04.992317 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:58:04.992566 (+ 249us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:58:04.992706 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0806 04:58:04.993078 (+ 372us) server_negotiation.cc:299] Negotiation successful
0806 04:58:04.993385 (+ 307us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":101,"server-negotiator.queue_time_us":69}
I20260806 04:58:04.998183 20064 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 50f00ca72a174ac4a6a7f39da3065462: Bootstrap starting.
I20260806 04:58:05.001251 20064 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 50f00ca72a174ac4a6a7f39da3065462: Neither blocks nor log segments found. Creating new log.
I20260806 04:58:05.002290 20064 log.cc:824] T 00000000000000000000000000000000 P 50f00ca72a174ac4a6a7f39da3065462: Log is configured to *not* fsync() on all Append() calls
I20260806 04:58:05.005211 20064 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 50f00ca72a174ac4a6a7f39da3065462: No bootstrap required, opened a new log
I20260806 04:58:05.012256 20064 raft_consensus.cc:348] T 00000000000000000000000000000000 P 50f00ca72a174ac4a6a7f39da3065462 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6126c11fed1e4765becce7bd422ce904" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 40031 } } peers { permanent_uuid: "89e9df10c70d4ddaacd2a30fff7a3b2d" member_type: VOTER last_known_addr { host: "127.9.96.125" port: 39617 } } peers { permanent_uuid: "50f00ca72a174ac4a6a7f39da3065462" member_type: VOTER last_known_addr { host: "127.9.96.124" port: 44477 } }
I20260806 04:58:05.012573 20064 raft_consensus.cc:374] T 00000000000000000000000000000000 P 50f00ca72a174ac4a6a7f39da3065462 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 04:58:05.012699 20064 raft_consensus.cc:729] T 00000000000000000000000000000000 P 50f00ca72a174ac4a6a7f39da3065462 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 50f00ca72a174ac4a6a7f39da3065462, State: Initialized, Role: FOLLOWER
I20260806 04:58:05.013269 20064 consensus_queue.cc:260] T 00000000000000000000000000000000 P 50f00ca72a174ac4a6a7f39da3065462 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6126c11fed1e4765becce7bd422ce904" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 40031 } } peers { permanent_uuid: "89e9df10c70d4ddaacd2a30fff7a3b2d" member_type: VOTER last_known_addr { host: "127.9.96.125" port: 39617 } } peers { permanent_uuid: "50f00ca72a174ac4a6a7f39da3065462" member_type: VOTER last_known_addr { host: "127.9.96.124" port: 44477 } }
I20260806 04:58:05.014160 20078 sys_catalog.cc:455] T 00000000000000000000000000000000 P 50f00ca72a174ac4a6a7f39da3065462 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6126c11fed1e4765becce7bd422ce904" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 40031 } } peers { permanent_uuid: "89e9df10c70d4ddaacd2a30fff7a3b2d" member_type: VOTER last_known_addr { host: "127.9.96.125" port: 39617 } } peers { permanent_uuid: "50f00ca72a174ac4a6a7f39da3065462" member_type: VOTER last_known_addr { host: "127.9.96.124" port: 44477 } } }
I20260806 04:58:05.014497 20078 sys_catalog.cc:458] T 00000000000000000000000000000000 P 50f00ca72a174ac4a6a7f39da3065462 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 04:58:05.015097 20064 sys_catalog.cc:564] T 00000000000000000000000000000000 P 50f00ca72a174ac4a6a7f39da3065462 [sys.catalog]: configured and running, proceeding with master startup.
W20260806 04:58:05.024662 20096 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 50f00ca72a174ac4a6a7f39da3065462: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 04:58:05.024798 20096 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 04:58:05.028957 19806 master_runner.cc:428] Detected that this master 50f00ca72a174ac4a6a7f39da3065462 is joining an existing cluster
I20260806 04:58:05.029081 19806 master_runner.cc:432] Initiating AddMaster RPC to add 127.9.96.124:44477
I20260806 04:58:05.052278 20074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:58:05.033177 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48738 (local address 127.9.96.126:40031)
0806 04:58:05.033453 (+ 276us) server_negotiation.cc:206] Beginning negotiation
0806 04:58:05.033461 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 04:58:05.035979 (+ 2518us) server_negotiation.cc:407] Connection header received
0806 04:58:05.036236 (+ 257us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:58:05.036241 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:58:05.036357 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:58:05.036443 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:58:05.037959 (+ 1516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:05.039041 (+ 1082us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:05.041208 (+ 2167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:05.041841 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:05.041946 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:58:05.045004 (+ 3058us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:58:05.045010 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:58:05.045015 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:58:05.045055 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:58:05.048857 (+ 3802us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:58:05.049797 (+ 940us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:58:05.049803 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:58:05.049807 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:58:05.049979 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:58:05.050491 (+ 512us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:58:05.050497 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:58:05.050500 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:58:05.050837 (+ 337us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:58:05.050987 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0806 04:58:05.051645 (+ 658us) server_negotiation.cc:299] Negotiation successful
0806 04:58:05.051888 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":130}
I20260806 04:58:05.052928 20069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:58:05.033328 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55168 (local address 127.9.96.124:44477)
0806 04:58:05.033515 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0806 04:58:05.033522 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 04:58:05.034397 (+ 875us) server_negotiation.cc:407] Connection header received
0806 04:58:05.034610 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:58:05.034615 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:58:05.034697 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:58:05.034803 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:58:05.036385 (+ 1582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:05.037917 (+ 1532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:05.040686 (+ 2769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:05.041663 (+ 977us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:05.041782 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:58:05.044907 (+ 3125us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:58:05.044915 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:58:05.044924 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:58:05.044967 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:58:05.049501 (+ 4534us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:58:05.050225 (+ 724us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:58:05.050232 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:58:05.050236 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:58:05.050410 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:58:05.050845 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:58:05.050849 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:58:05.050854 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:58:05.051315 (+ 461us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:58:05.051502 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0806 04:58:05.052105 (+ 603us) server_negotiation.cc:299] Negotiation successful
0806 04:58:05.052542 (+ 437us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":40}
I20260806 04:58:05.053434 20105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:58:05.033619 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:40031 (local address 127.0.0.1:48738)
0806 04:58:05.035878 (+ 2259us) negotiation.cc:107] Waiting for socket to connect
0806 04:58:05.035895 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 04:58:05.036077 (+ 182us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:58:05.036666 (+ 589us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:58:05.036671 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:58:05.037357 (+ 686us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:58:05.037771 (+ 414us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:58:05.037785 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:58:05.039255 (+ 1470us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:58:05.039260 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:58:05.041021 (+ 1761us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:58:05.041036 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:58:05.042239 (+ 1203us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:58:05.042246 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:58:05.042433 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:58:05.043340 (+ 907us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:58:05.043393 (+ 53us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:58:05.044781 (+ 1388us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 04:58:05.049022 (+ 4241us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:58:05.049028 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:58:05.049032 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:58:05.049655 (+ 623us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:58:05.050132 (+ 477us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:58:05.050137 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:58:05.050141 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:58:05.050331 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:58:05.051077 (+ 746us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 04:58:05.051082 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 04:58:05.051314 (+ 232us) client_negotiation.cc:769] Sending connection context
0806 04:58:05.052505 (+ 1191us) client_negotiation.cc:240] Negotiation successful
0806 04:58:05.052829 (+ 324us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2126,"mutex_wait_us":29,"thread_start_us":1951,"threads_started":1}
I20260806 04:58:05.054167 20104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:58:05.033431 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.124:44477 (local address 127.0.0.1:55168)
0806 04:58:05.034260 (+ 829us) negotiation.cc:107] Waiting for socket to connect
0806 04:58:05.034285 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0806 04:58:05.034455 (+ 170us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:58:05.034864 (+ 409us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:58:05.034871 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:58:05.035821 (+ 950us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:58:05.036233 (+ 412us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:58:05.036249 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:58:05.038189 (+ 1940us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:58:05.038196 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:58:05.040493 (+ 2297us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:58:05.040517 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:58:05.042295 (+ 1778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:58:05.042301 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:58:05.042565 (+ 264us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:58:05.043186 (+ 621us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:58:05.043209 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:58:05.044704 (+ 1495us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 04:58:05.049631 (+ 4927us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:58:05.049635 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:58:05.049638 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:58:05.050101 (+ 463us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:58:05.050538 (+ 437us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:58:05.050542 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:58:05.050546 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:58:05.050716 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:58:05.051524 (+ 808us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 04:58:05.051528 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 04:58:05.051824 (+ 296us) client_negotiation.cc:769] Sending connection context
0806 04:58:05.053274 (+ 1450us) client_negotiation.cc:240] Negotiation successful
0806 04:58:05.053715 (+ 441us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":636,"mutex_wait_us":25,"thread_start_us":458,"threads_started":1}
I20260806 04:58:05.061014 19981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:58:05.036161 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53010 (local address 127.9.96.125:39617)
0806 04:58:05.036369 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0806 04:58:05.036375 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 04:58:05.043200 (+ 6825us) server_negotiation.cc:407] Connection header received
0806 04:58:05.043626 (+ 426us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:58:05.043632 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:58:05.043717 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:58:05.043795 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:58:05.046151 (+ 2356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:05.047319 (+ 1168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:05.049632 (+ 2313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:05.050326 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:05.050422 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:58:05.054127 (+ 3705us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:58:05.054134 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:58:05.054138 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:58:05.054178 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:58:05.057854 (+ 3676us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:58:05.058626 (+ 772us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:58:05.058633 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:58:05.058639 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:58:05.058806 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:58:05.059243 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:58:05.059248 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:58:05.059250 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:58:05.059704 (+ 454us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:58:05.059865 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0806 04:58:05.060402 (+ 537us) server_negotiation.cc:299] Negotiation successful
0806 04:58:05.060650 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260806 04:58:05.061426 20106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:58:05.036500 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.125:39617 (local address 127.0.0.1:53010)
0806 04:58:05.043096 (+ 6596us) negotiation.cc:107] Waiting for socket to connect
0806 04:58:05.043117 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0806 04:58:05.043442 (+ 325us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:58:05.044021 (+ 579us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:58:05.044026 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:58:05.045465 (+ 1439us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:58:05.045943 (+ 478us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:58:05.045959 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:58:05.047607 (+ 1648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:58:05.047615 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:58:05.049465 (+ 1850us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:58:05.049480 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:58:05.050708 (+ 1228us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:58:05.050713 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:58:05.050901 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:58:05.051837 (+ 936us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:58:05.051859 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:58:05.053503 (+ 1644us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 04:58:05.058001 (+ 4498us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:58:05.058006 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:58:05.058010 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:58:05.058504 (+ 494us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:58:05.058941 (+ 437us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:58:05.058946 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:58:05.058949 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:58:05.059115 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:58:05.059839 (+ 724us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 04:58:05.059844 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 04:58:05.060076 (+ 232us) client_negotiation.cc:769] Sending connection context
0806 04:58:05.060293 (+ 217us) client_negotiation.cc:240] Negotiation successful
0806 04:58:05.060613 (+ 320us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6427,"thread_start_us":6718,"threads_started":1}
Aug 06 04:58:05 dist-test-slave-x4qh krb5kdc[18896](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992282, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.96.126@KRBTEST.COM for oryx/127.9.96.124@KRBTEST.COM
I20260806 04:58:05.070665 9601 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu pid 19048
I20260806 04:58:05.075289 19978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:58:05.059326 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.124:44477 (local address 127.0.0.1:55172)
0806 04:58:05.059717 (+ 391us) negotiation.cc:107] Waiting for socket to connect
0806 04:58:05.059736 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0806 04:58:05.059833 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:58:05.060908 (+ 1075us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:58:05.060913 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:58:05.061388 (+ 475us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:58:05.061566 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:58:05.061582 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:58:05.063026 (+ 1444us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:58:05.063030 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:58:05.063956 (+ 926us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:58:05.063968 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:58:05.064939 (+ 971us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:58:05.064943 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:58:05.065058 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:58:05.065594 (+ 536us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:58:05.065609 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:58:05.068079 (+ 2470us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 04:58:05.071795 (+ 3716us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:58:05.071800 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:58:05.071805 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:58:05.072431 (+ 626us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:58:05.072969 (+ 538us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:58:05.072974 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:58:05.072978 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:58:05.073135 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:58:05.074031 (+ 896us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 04:58:05.074036 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 04:58:05.074242 (+ 206us) client_negotiation.cc:769] Sending connection context
0806 04:58:05.074400 (+ 158us) client_negotiation.cc:240] Negotiation successful
0806 04:58:05.074660 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":269}
I20260806 04:58:05.075425 20069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:58:05.060142 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55172 (local address 127.9.96.124:44477)
0806 04:58:05.060341 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0806 04:58:05.060346 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 04:58:05.060373 (+ 27us) server_negotiation.cc:407] Connection header received
0806 04:58:05.060459 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:58:05.060463 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:58:05.060581 (+ 118us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:58:05.060681 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:58:05.061730 (+ 1049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:05.062891 (+ 1161us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:05.064101 (+ 1210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:05.064801 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:05.064944 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:58:05.068252 (+ 3308us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:58:05.068256 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:58:05.068259 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:58:05.068286 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:58:05.071609 (+ 3323us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:58:05.072628 (+ 1019us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:58:05.072636 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:58:05.072641 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:58:05.072814 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:58:05.073393 (+ 579us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:58:05.073422 (+ 29us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:58:05.073428 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:58:05.073889 (+ 461us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:58:05.074071 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0806 04:58:05.074552 (+ 481us) server_negotiation.cc:299] Negotiation successful
0806 04:58:05.074850 (+ 298us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260806 04:58:05.077533 18960 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.9.96.124:44477: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "50f00ca72a174ac4a6a7f39da3065462"
member_type: NON_VOTER
last_known_addr {
host: "127.9.96.124"
port: 44477
}
attrs {
promote: true
}
}
dest_uuid: "6126c11fed1e4765becce7bd422ce904"
cas_config_opid_index: 6
I20260806 04:58:05.078565 18960 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6126c11fed1e4765becce7bd422ce904 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "6126c11fed1e4765becce7bd422ce904" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 40031 } } peers { permanent_uuid: "89e9df10c70d4ddaacd2a30fff7a3b2d" member_type: VOTER last_known_addr { host: "127.9.96.125" port: 39617 } attrs { promote: false } } peers { permanent_uuid: "50f00ca72a174ac4a6a7f39da3065462" member_type: NON_VOTER last_known_addr { host: "127.9.96.124" port: 44477 } attrs { promote: true } }
I20260806 04:58:05.081102 20018 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 50f00ca72a174ac4a6a7f39da3065462 [term 0 FOLLOWER]: Advancing to term 1
I20260806 04:58:05.084101 20018 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 50f00ca72a174ac4a6a7f39da3065462 [term 1 FOLLOWER]: Refusing update from remote peer 6126c11fed1e4765becce7bd422ce904: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260806 04:58:05.084853 19885 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 89e9df10c70d4ddaacd2a30fff7a3b2d [term 1 FOLLOWER]: Refusing update from remote peer 6126c11fed1e4765becce7bd422ce904: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260806 04:58:05.085521 19975 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 6126c11fed1e4765becce7bd422ce904 [LEADER]: Connected to new peer: Peer: permanent_uuid: "89e9df10c70d4ddaacd2a30fff7a3b2d" member_type: VOTER last_known_addr { host: "127.9.96.125" port: 39617 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260806 04:58:05.086205 19975 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 6126c11fed1e4765becce7bd422ce904 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50f00ca72a174ac4a6a7f39da3065462" member_type: NON_VOTER last_known_addr { host: "127.9.96.124" port: 44477 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 04:58:05.089942 19974 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 6126c11fed1e4765becce7bd422ce904 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER 50f00ca72a174ac4a6a7f39da3065462 (127.9.96.124) added. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "6126c11fed1e4765becce7bd422ce904" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 40031 } } peers { permanent_uuid: "89e9df10c70d4ddaacd2a30fff7a3b2d" member_type: VOTER last_known_addr { host: "127.9.96.125" port: 39617 } attrs { promote: false } } peers { permanent_uuid: "50f00ca72a174ac4a6a7f39da3065462" member_type: NON_VOTER last_known_addr { host: "127.9.96.124" port: 44477 } attrs { promote: true } } }
Aug 06 04:58:05 dist-test-slave-x4qh krb5kdc[18896](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992281, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.9.96.65@KRBTEST.COM
I20260806 04:58:05.090543 19885 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 89e9df10c70d4ddaacd2a30fff7a3b2d [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER 50f00ca72a174ac4a6a7f39da3065462 (127.9.96.124) added. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "6126c11fed1e4765becce7bd422ce904" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 40031 } } peers { permanent_uuid: "89e9df10c70d4ddaacd2a30fff7a3b2d" member_type: VOTER last_known_addr { host: "127.9.96.125" port: 39617 } attrs { promote: false } } peers { permanent_uuid: "50f00ca72a174ac4a6a7f39da3065462" member_type: NON_VOTER last_known_addr { host: "127.9.96.124" port: 44477 } attrs { promote: true } } }
I20260806 04:58:05.094048 19974 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.9.96.124:44477
I20260806 04:58:05.094388 19974 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6126c11fed1e4765becce7bd422ce904 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "6126c11fed1e4765becce7bd422ce904" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "6126c11fed1e4765becce7bd422ce904" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 40031 } } peers { permanent_uuid: "89e9df10c70d4ddaacd2a30fff7a3b2d" member_type: VOTER last_known_addr { host: "127.9.96.125" port: 39617 } attrs { promote: false } } peers { permanent_uuid: "50f00ca72a174ac4a6a7f39da3065462" member_type: NON_VOTER last_known_addr { host: "127.9.96.124" port: 44477 } attrs { promote: true } } }
I20260806 04:58:05.094662 19974 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6126c11fed1e4765becce7bd422ce904 [sys.catalog]: This master's current role is: LEADER
I20260806 04:58:05.094524 19929 sys_catalog.cc:455] T 00000000000000000000000000000000 P 89e9df10c70d4ddaacd2a30fff7a3b2d [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "6126c11fed1e4765becce7bd422ce904" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "6126c11fed1e4765becce7bd422ce904" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 40031 } } peers { permanent_uuid: "89e9df10c70d4ddaacd2a30fff7a3b2d" member_type: VOTER last_known_addr { host: "127.9.96.125" port: 39617 } attrs { promote: false } } peers { permanent_uuid: "50f00ca72a174ac4a6a7f39da3065462" member_type: NON_VOTER last_known_addr { host: "127.9.96.124" port: 44477 } attrs { promote: true } } }
I20260806 04:58:05.094826 19929 sys_catalog.cc:458] T 00000000000000000000000000000000 P 89e9df10c70d4ddaacd2a30fff7a3b2d [sys.catalog]: This master's current role is: FOLLOWER
I20260806 04:58:05.095257 19806 master.cc:537] Master@127.9.96.124:44477 shutting down...
I20260806 04:58:05.098291 19802 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6126c11fed1e4765becce7bd422ce904 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "6126c11fed1e4765becce7bd422ce904" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "6126c11fed1e4765becce7bd422ce904" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 40031 } } peers { permanent_uuid: "89e9df10c70d4ddaacd2a30fff7a3b2d" member_type: VOTER last_known_addr { host: "127.9.96.125" port: 39617 } attrs { promote: false } } peers { permanent_uuid: "50f00ca72a174ac4a6a7f39da3065462" member_type: NON_VOTER last_known_addr { host: "127.9.96.124" port: 44477 } attrs { promote: true } } }
I20260806 04:58:05.098613 19802 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6126c11fed1e4765becce7bd422ce904 [sys.catalog]: This master's current role is: LEADER
I20260806 04:58:05.101441 20018 raft_consensus.cc:858] T 00000000000000000000000000000000 P 50f00ca72a174ac4a6a7f39da3065462 [term 1 NON_PARTICIPANT]: Illegal state: RaftConfig change currently pending. Only one is allowed at a time.
Committed config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6126c11fed1e4765becce7bd422ce904" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 40031 } } peers { permanent_uuid: "89e9df10c70d4ddaacd2a30fff7a3b2d" member_type: VOTER last_known_addr { host: "127.9.96.125" port: 39617 } } peers { permanent_uuid: "50f00ca72a174ac4a6a7f39da3065462" member_type: VOTER last_known_addr { host: "127.9.96.124" port: 44477 } }.
Pending config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "6126c11fed1e4765becce7bd422ce904" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 40031 } } peers { permanent_uuid: "89e9df10c70d4ddaacd2a30fff7a3b2d" member_type: NON_VOTER last_known_addr { host: "127.9.96.125" port: 39617 } attrs { promote: true } }:
New config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "6126c11fed1e4765becce7bd422ce904" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 40031 } } peers { permanent_uuid: "89e9df10c70d4ddaacd2a30fff7a3b2d" member_type: VOTER last_known_addr { host: "127.9.96.125" port: 39617 } attrs { promote: false } }
W20260806 04:58:05.101611 20018 raft_consensus.cc:1565] T 00000000000000000000000000000000 P 50f00ca72a174ac4a6a7f39da3065462 [term 1 NON_PARTICIPANT]: Could not prepare op 'term: 1 index: 6' and following 1 ops. Status for this op: Illegal state: RaftConfig change currently pending. Only one is allowed at a time.
Committed config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6126c11fed1e4765becce7bd422ce904" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 40031 } } peers { permanent_uuid: "89e9df10c70d4ddaacd2a30fff7a3b2d" member_type: VOTER last_known_addr { host: "127.9.96.125" port: 39617 } } peers { permanent_uuid: "50f00ca72a174ac4a6a7f39da3065462" member_type: VOTER last_known_addr { host: "127.9.96.124" port: 44477 } }.
Pending config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "6126c11fed1e4765becce7bd422ce904" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 40031 } } peers { permanent_uuid: "89e9df10c70d4ddaacd2a30fff7a3b2d" member_type: NON_VOTER last_known_addr { host: "127.9.96.125" port: 39617 } attrs { promote: true } }:
New config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "6126c11fed1e4765becce7bd422ce904" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 40031 } } peers { permanent_uuid: "89e9df10c70d4ddaacd2a30fff7a3b2d" member_type: VOTER last_known_addr { host: "127.9.96.125" port: 39617 } attrs { promote: false } }
I20260806 04:58:05.101966 20107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:58:05.071919 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56352 (local address 127.9.96.65:37807)
0806 04:58:05.079619 (+ 7700us) server_negotiation.cc:206] Beginning negotiation
0806 04:58:05.079629 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 04:58:05.079661 (+ 32us) server_negotiation.cc:407] Connection header received
0806 04:58:05.079737 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:58:05.079748 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:58:05.080029 (+ 281us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:58:05.082011 (+ 1982us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:58:05.082253 (+ 242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:05.083947 (+ 1694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:05.085612 (+ 1665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:05.086473 (+ 861us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:05.086630 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:58:05.092163 (+ 5533us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:58:05.092176 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:58:05.092199 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:58:05.092239 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:58:05.096967 (+ 4728us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:58:05.097873 (+ 906us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:58:05.097882 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:58:05.097886 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:58:05.098049 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:58:05.098532 (+ 483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:58:05.098538 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:58:05.098641 (+ 103us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:58:05.099349 (+ 708us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:58:05.100841 (+ 1492us) server_negotiation.cc:1035] Waiting for connection context
0806 04:58:05.101026 (+ 185us) server_negotiation.cc:299] Negotiation successful
0806 04:58:05.101307 (+ 281us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7412,"thread_start_us":7242,"threads_started":1}
I20260806 04:58:05.103323 19226 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 04:58:05.121141 20018 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 50f00ca72a174ac4a6a7f39da3065462 [term 1 NON_PARTICIPANT]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 89e9df10c70d4ddaacd2a30fff7a3b2d (127.9.96.125) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "6126c11fed1e4765becce7bd422ce904" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 40031 } } peers { permanent_uuid: "89e9df10c70d4ddaacd2a30fff7a3b2d" member_type: NON_VOTER last_known_addr { host: "127.9.96.125" port: 39617 } attrs { promote: true } } }
I20260806 04:58:05.126940 20114 sys_catalog.cc:455] T 00000000000000000000000000000000 P 50f00ca72a174ac4a6a7f39da3065462 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "6126c11fed1e4765becce7bd422ce904" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "6126c11fed1e4765becce7bd422ce904" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 40031 } } peers { permanent_uuid: "89e9df10c70d4ddaacd2a30fff7a3b2d" member_type: NON_VOTER last_known_addr { host: "127.9.96.125" port: 39617 } attrs { promote: true } } }
I20260806 04:58:05.127275 20114 sys_catalog.cc:458] T 00000000000000000000000000000000 P 50f00ca72a174ac4a6a7f39da3065462 [sys.catalog]: This master's current role is: NON_PARTICIPANT
I20260806 04:58:05.127646 20078 sys_catalog.cc:455] T 00000000000000000000000000000000 P 50f00ca72a174ac4a6a7f39da3065462 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6126c11fed1e4765becce7bd422ce904. Latest consensus state: current_term: 1 leader_uuid: "6126c11fed1e4765becce7bd422ce904" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "6126c11fed1e4765becce7bd422ce904" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 40031 } } peers { permanent_uuid: "89e9df10c70d4ddaacd2a30fff7a3b2d" member_type: NON_VOTER last_known_addr { host: "127.9.96.125" port: 39617 } attrs { promote: true } } }
I20260806 04:58:05.127842 20078 sys_catalog.cc:458] T 00000000000000000000000000000000 P 50f00ca72a174ac4a6a7f39da3065462 [sys.catalog]: This master's current role is: NON_PARTICIPANT
I20260806 04:58:05.127770 19974 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6126c11fed1e4765becce7bd422ce904 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "6126c11fed1e4765becce7bd422ce904" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "6126c11fed1e4765becce7bd422ce904" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 40031 } } peers { permanent_uuid: "89e9df10c70d4ddaacd2a30fff7a3b2d" member_type: VOTER last_known_addr { host: "127.9.96.125" port: 39617 } attrs { promote: false } } peers { permanent_uuid: "50f00ca72a174ac4a6a7f39da3065462" member_type: NON_VOTER last_known_addr { host: "127.9.96.124" port: 44477 } attrs { promote: true } } }
I20260806 04:58:05.128067 19974 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6126c11fed1e4765becce7bd422ce904 [sys.catalog]: This master's current role is: LEADER
W20260806 04:58:05.129506 18937 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
I20260806 04:58:05.128522 20118 sys_catalog.cc:455] T 00000000000000000000000000000000 P 50f00ca72a174ac4a6a7f39da3065462 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "6126c11fed1e4765becce7bd422ce904" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "6126c11fed1e4765becce7bd422ce904" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 40031 } } peers { permanent_uuid: "89e9df10c70d4ddaacd2a30fff7a3b2d" member_type: NON_VOTER last_known_addr { host: "127.9.96.125" port: 39617 } attrs { promote: true } } }
I20260806 04:58:05.133152 20118 sys_catalog.cc:458] T 00000000000000000000000000000000 P 50f00ca72a174ac4a6a7f39da3065462 [sys.catalog]: This master's current role is: NON_PARTICIPANT
I20260806 04:58:05.136241 19806 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 50f00ca72a174ac4a6a7f39da3065462 [term 1 NON_PARTICIPANT]: Raft consensus shutting down.
W20260806 04:58:05.136279 18937 consensus_peers.cc:487] T 00000000000000000000000000000000 P 6126c11fed1e4765becce7bd422ce904 -> Peer 50f00ca72a174ac4a6a7f39da3065462 (127.9.96.124:44477): Couldn't send request to peer 50f00ca72a174ac4a6a7f39da3065462. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260806 04:58:05.136677 19806 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 50f00ca72a174ac4a6a7f39da3065462 [term 1 NON_PARTICIPANT]: Raft consensus is shut down!
I20260806 04:58:05.136847 19806 tablet_replica.cc:330] T 00000000000000000000000000000000 P 50f00ca72a174ac4a6a7f39da3065462: stopping tablet replica
I20260806 04:58:05.145349 19806 master.cc:552] Master@127.9.96.124:44477 shutdown complete.
I20260806 04:58:05.145442 19806 master_runner.cc:305] Clearing existing system tablet
I20260806 04:58:05.145825 19806 fs_manager.cc:359] Metadata directory not provided
I20260806 04:58:05.145927 19806 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 04:58:05.148882 19806 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.000s sys 0.002s
I20260806 04:58:05.150498 20121 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 04:58:05.150915 19806 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785992227050689-9601-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 04:58:05.151153 19806 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.002s
I20260806 04:58:05.151247 19806 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785992227050689-9601-0/minicluster-data/master-2/data,/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785992227050689-9601-0/minicluster-data/master-2/wal
uuid: "50f00ca72a174ac4a6a7f39da3065462"
format_stamp: "Formatted at 2026-08-06 04:58:04 on dist-test-slave-x4qh"
server_key: "74e377d6477d8894f812546c5013ec32"
server_key_iv: "ef898b742ca2a6e0a0d35d36d52dee79"
server_key_version: "encryptionkey@0"
I20260806 04:58:05.155845 19806 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 50f00ca72a174ac4a6a7f39da3065462: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 04:58:05.162475 19806 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 50f00ca72a174ac4a6a7f39da3065462: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260806 04:58:05.162619 19806 log.cc:1196] T 00000000000000000000000000000000 P 50f00ca72a174ac4a6a7f39da3065462: Deleting WAL directory at /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785992227050689-9601-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260806 04:58:05.163094 19806 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 50f00ca72a174ac4a6a7f39da3065462: Deleting consensus metadata
I20260806 04:58:05.163335 19806 master_runner.cc:315] Copying system tablet from 127.9.96.126:40031
I20260806 04:58:05.166661 19806 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 50f00ca72a174ac4a6a7f39da3065462: tablet copy: Beginning tablet copy session from remote peer at address 127.9.96.126:40031
I20260806 04:58:05.190531 20074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:58:05.167545 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48742 (local address 127.9.96.126:40031)
0806 04:58:05.167723 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0806 04:58:05.167729 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 04:58:05.175945 (+ 8216us) server_negotiation.cc:407] Connection header received
0806 04:58:05.176130 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:58:05.176136 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:58:05.176236 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:58:05.176361 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:58:05.177416 (+ 1055us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:05.178564 (+ 1148us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:05.180506 (+ 1942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:05.181168 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:05.181308 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:58:05.183633 (+ 2325us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:58:05.183640 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:58:05.183646 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:58:05.183686 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:58:05.187518 (+ 3832us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:58:05.188307 (+ 789us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:58:05.188313 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:58:05.188317 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:58:05.188489 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:58:05.188923 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:58:05.188928 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:58:05.188931 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:58:05.189265 (+ 334us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:58:05.189540 (+ 275us) server_negotiation.cc:1035] Waiting for connection context
0806 04:58:05.189897 (+ 357us) server_negotiation.cc:299] Negotiation successful
0806 04:58:05.190145 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260806 04:58:05.190872 20130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:58:05.167399 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:40031 (local address 127.0.0.1:48742)
0806 04:58:05.175805 (+ 8406us) negotiation.cc:107] Waiting for socket to connect
0806 04:58:05.175826 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0806 04:58:05.175981 (+ 155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:58:05.176416 (+ 435us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:58:05.176421 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:58:05.176961 (+ 540us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:58:05.177245 (+ 284us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:58:05.177257 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:58:05.178726 (+ 1469us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:58:05.178732 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:58:05.180333 (+ 1601us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:58:05.180348 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:58:05.181333 (+ 985us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:58:05.181338 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:58:05.181524 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:58:05.182234 (+ 710us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:58:05.182260 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:58:05.183410 (+ 1150us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 04:58:05.187655 (+ 4245us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:58:05.187661 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:58:05.187665 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:58:05.188160 (+ 495us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:58:05.188604 (+ 444us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:58:05.188609 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:58:05.188613 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:58:05.188786 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:58:05.189395 (+ 609us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 04:58:05.189406 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 04:58:05.189592 (+ 186us) client_negotiation.cc:769] Sending connection context
0806 04:58:05.189757 (+ 165us) client_negotiation.cc:240] Negotiation successful
0806 04:58:05.190099 (+ 342us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8232,"thread_start_us":8115,"threads_started":1}
I20260806 04:58:05.191579 18990 tablet_copy_service.cc:140] P 6126c11fed1e4765becce7bd422ce904: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 50f00ca72a174ac4a6a7f39da3065462 ({username='oryx', principal='oryx/127.9.96.124@KRBTEST.COM'} at 127.0.0.1:48742)
I20260806 04:58:05.191749 18990 tablet_copy_service.cc:161] P 6126c11fed1e4765becce7bd422ce904: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 50f00ca72a174ac4a6a7f39da3065462 at {username='oryx', principal='oryx/127.9.96.124@KRBTEST.COM'} at 127.0.0.1:48742: session id = 50f00ca72a174ac4a6a7f39da3065462-00000000000000000000000000000000
I20260806 04:58:05.195415 18990 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 6126c11fed1e4765becce7bd422ce904: Tablet Copy: opened 0 blocks and 1 log segments
I20260806 04:58:05.197376 19806 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260806 04:58:05.204177 19806 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 50f00ca72a174ac4a6a7f39da3065462: tablet copy: Starting download of 0 data blocks...
I20260806 04:58:05.204533 19806 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 50f00ca72a174ac4a6a7f39da3065462: tablet copy: Starting download of 1 WAL segments...
I20260806 04:58:05.207043 19806 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 50f00ca72a174ac4a6a7f39da3065462: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260806 04:58:05.211334 18990 tablet_copy_service.cc:342] P 6126c11fed1e4765becce7bd422ce904: Request end of tablet copy session 50f00ca72a174ac4a6a7f39da3065462-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.9.96.124@KRBTEST.COM'} at 127.0.0.1:48742
I20260806 04:58:05.211504 18990 tablet_copy_service.cc:433] P 6126c11fed1e4765becce7bd422ce904: ending tablet copy session 50f00ca72a174ac4a6a7f39da3065462-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 50f00ca72a174ac4a6a7f39da3065462
I20260806 04:58:05.222528 19806 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 04:58:05.225167 20131 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:58:05.228761 20133 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:58:05.236248 20137 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:58:05.237636 9601 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu with pid 19048
I20260806 04:58:05.244810 19806 server_base.cc:1034] running on GCE node
I20260806 04:58:05.245218 19806 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 04:58:05.251693 19806 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 04:58:05.252981 19806 hybrid_clock.cc:648] HybridClock initialized: now 1785992285252945 us; error 32 us; skew 500 ppm
I20260806 04:58:05.253247 19806 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 04:58:05 dist-test-slave-x4qh krb5kdc[18896](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992285, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.96.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 04:58:05.255218 19806 init.cc:377] Logged in from keytab as oryx/127.9.96.124@KRBTEST.COM (short username oryx)
I20260806 04:58:05.259212 19806 webserver.cc:460] Webserver started at http://127.9.96.124:45197/ using document root <none> and password file <none>
I20260806 04:58:05.259570 19806 fs_manager.cc:359] Metadata directory not provided
I20260806 04:58:05.259687 19806 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 04:58:05.262408 19806 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20260806 04:58:05.264227 20147 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 04:58:05.264727 19806 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.001s
I20260806 04:58:05.264878 19806 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785992227050689-9601-0/minicluster-data/master-2/data,/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785992227050689-9601-0/minicluster-data/master-2/wal
uuid: "50f00ca72a174ac4a6a7f39da3065462"
format_stamp: "Formatted at 2026-08-06 04:58:04 on dist-test-slave-x4qh"
server_key: "74e377d6477d8894f812546c5013ec32"
server_key_iv: "ef898b742ca2a6e0a0d35d36d52dee79"
server_key_version: "encryptionkey@0"
I20260806 04:58:05.265079 19806 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785992227050689-9601-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785992227050689-9601-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785992227050689-9601-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 04:58:05.273067 9601 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
/tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785992227050689-9601-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785992227050689-9601-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785992227050689-9601-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785992227050689-9601-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.96.65:37807
--local_ip_for_outbound_sockets=127.9.96.65
--tserver_master_addrs=127.9.96.126:40031
--webserver_port=37575
--webserver_interface=127.9.96.65
--keytab_file=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785992227050689-9601-0/krb5kdc/oryx_127.9.96.65.keytab
--principal=oryx/127.9.96.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.96.84:34171
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.9.96.126:40031,127.9.96.125:39617
--tserver_master_addrs=127.9.96.126:40031,127.9.96.125:39617,127.9.96.124:44477 with env {KRB5CCNAME=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785992227050689-9601-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785992227050689-9601-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785992227050689-9601-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 04:58:05.277299 19806 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 04:58:05.281847 19806 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 04:58:05.322515 19806 rpc_server.cc:307] RPC server started. Bound to: 127.9.96.124:44477
I20260806 04:58:05.322558 20232 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.96.124:44477 every 8 connection(s)
I20260806 04:58:05.323523 19806 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785992227050689-9601-0/minicluster-data/master-2/data/info.pb
I20260806 04:58:05.330845 20233 sys_catalog.cc:263] Verifying existing consensus state
I20260806 04:58:05.336045 20233 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 50f00ca72a174ac4a6a7f39da3065462: Bootstrap starting.
I20260806 04:58:05.350054 20233 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 50f00ca72a174ac4a6a7f39da3065462: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 04:58:05.350533 20233 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 50f00ca72a174ac4a6a7f39da3065462: Bootstrap complete.
I20260806 04:58:05.351464 20233 raft_consensus.cc:348] T 00000000000000000000000000000000 P 50f00ca72a174ac4a6a7f39da3065462 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "6126c11fed1e4765becce7bd422ce904" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 40031 } } peers { permanent_uuid: "89e9df10c70d4ddaacd2a30fff7a3b2d" member_type: VOTER last_known_addr { host: "127.9.96.125" port: 39617 } attrs { promote: false } } peers { permanent_uuid: "50f00ca72a174ac4a6a7f39da3065462" member_type: NON_VOTER last_known_addr { host: "127.9.96.124" port: 44477 } attrs { promote: true } }
I20260806 04:58:05.351727 20233 raft_consensus.cc:729] T 00000000000000000000000000000000 P 50f00ca72a174ac4a6a7f39da3065462 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 50f00ca72a174ac4a6a7f39da3065462, State: Initialized, Role: LEARNER
I20260806 04:58:05.351958 20233 consensus_queue.cc:260] T 00000000000000000000000000000000 P 50f00ca72a174ac4a6a7f39da3065462 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "6126c11fed1e4765becce7bd422ce904" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 40031 } } peers { permanent_uuid: "89e9df10c70d4ddaacd2a30fff7a3b2d" member_type: VOTER last_known_addr { host: "127.9.96.125" port: 39617 } attrs { promote: false } } peers { permanent_uuid: "50f00ca72a174ac4a6a7f39da3065462" member_type: NON_VOTER last_known_addr { host: "127.9.96.124" port: 44477 } attrs { promote: true } }
I20260806 04:58:05.355906 20236 sys_catalog.cc:455] T 00000000000000000000000000000000 P 50f00ca72a174ac4a6a7f39da3065462 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "6126c11fed1e4765becce7bd422ce904" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 40031 } } peers { permanent_uuid: "89e9df10c70d4ddaacd2a30fff7a3b2d" member_type: VOTER last_known_addr { host: "127.9.96.125" port: 39617 } attrs { promote: false } } peers { permanent_uuid: "50f00ca72a174ac4a6a7f39da3065462" member_type: NON_VOTER last_known_addr { host: "127.9.96.124" port: 44477 } attrs { promote: true } } }
I20260806 04:58:05.356235 20233 sys_catalog.cc:564] T 00000000000000000000000000000000 P 50f00ca72a174ac4a6a7f39da3065462 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 04:58:05.356290 20236 sys_catalog.cc:458] T 00000000000000000000000000000000 P 50f00ca72a174ac4a6a7f39da3065462 [sys.catalog]: This master's current role is: LEARNER
I20260806 04:58:05.364883 20254 catalog_manager.cc:1259] Loaded cluster ID: b1cd49f4c4814e84b0f51af05de8eeee
I20260806 04:58:05.365059 20254 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 50f00ca72a174ac4a6a7f39da3065462: loading cluster ID for follower catalog manager: success
I20260806 04:58:05.368208 20254 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 50f00ca72a174ac4a6a7f39da3065462: acquiring CA information for follower catalog manager: success
I20260806 04:58:05.370122 20254 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 50f00ca72a174ac4a6a7f39da3065462: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 04:58:05.627480 19802 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6126c11fed1e4765becce7bd422ce904 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "6126c11fed1e4765becce7bd422ce904" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "6126c11fed1e4765becce7bd422ce904" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 40031 } } peers { permanent_uuid: "89e9df10c70d4ddaacd2a30fff7a3b2d" member_type: VOTER last_known_addr { host: "127.9.96.125" port: 39617 } attrs { promote: false } } peers { permanent_uuid: "50f00ca72a174ac4a6a7f39da3065462" member_type: NON_VOTER last_known_addr { host: "127.9.96.124" port: 44477 } attrs { promote: true } } }
I20260806 04:58:05.627904 19802 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6126c11fed1e4765becce7bd422ce904 [sys.catalog]: This master's current role is: LEADER
I20260806 04:58:05.634163 20256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:58:05.628136 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55182 (local address 127.9.96.124:44477)
0806 04:58:05.628911 (+ 775us) server_negotiation.cc:206] Beginning negotiation
0806 04:58:05.628916 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 04:58:05.628931 (+ 15us) server_negotiation.cc:407] Connection header received
0806 04:58:05.628991 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:58:05.628995 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:58:05.629062 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:58:05.629188 (+ 126us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 04:58:05.629688 (+ 500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:05.630676 (+ 988us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:05.632265 (+ 1589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:05.633075 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:05.633231 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:58:05.633491 (+ 260us) server_negotiation.cc:1035] Waiting for connection context
0806 04:58:05.633676 (+ 185us) server_negotiation.cc:299] Negotiation successful
0806 04:58:05.633808 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":585,"thread_start_us":407,"threads_started":1}
I20260806 04:58:05.634162 20255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:58:05.627879 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.124:44477 (local address 127.0.0.1:55182)
0806 04:58:05.628523 (+ 644us) negotiation.cc:107] Waiting for socket to connect
0806 04:58:05.628547 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0806 04:58:05.628715 (+ 168us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:58:05.629217 (+ 502us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:58:05.629223 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:58:05.629266 (+ 43us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 04:58:05.629528 (+ 262us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:58:05.629538 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:58:05.630827 (+ 1289us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:58:05.630833 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:58:05.632112 (+ 1279us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:58:05.632125 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:58:05.633287 (+ 1162us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:58:05.633293 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:58:05.633486 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:58:05.633531 (+ 45us) client_negotiation.cc:769] Sending connection context
0806 04:58:05.633649 (+ 118us) client_negotiation.cc:240] Negotiation successful
0806 04:58:05.633789 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":514,"thread_start_us":391,"threads_started":1}
I20260806 04:58:05.635135 20194 raft_consensus.cc:1203] T 00000000000000000000000000000000 P 50f00ca72a174ac4a6a7f39da3065462 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.5->[1.6-1.7] Dedup: 1.7->[]
I20260806 04:58:05.635609 20236 sys_catalog.cc:455] T 00000000000000000000000000000000 P 50f00ca72a174ac4a6a7f39da3065462 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6126c11fed1e4765becce7bd422ce904. Latest consensus state: current_term: 1 leader_uuid: "6126c11fed1e4765becce7bd422ce904" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "6126c11fed1e4765becce7bd422ce904" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 40031 } } peers { permanent_uuid: "89e9df10c70d4ddaacd2a30fff7a3b2d" member_type: VOTER last_known_addr { host: "127.9.96.125" port: 39617 } attrs { promote: false } } peers { permanent_uuid: "50f00ca72a174ac4a6a7f39da3065462" member_type: NON_VOTER last_known_addr { host: "127.9.96.124" port: 44477 } attrs { promote: true } } }
I20260806 04:58:05.635892 20236 sys_catalog.cc:458] T 00000000000000000000000000000000 P 50f00ca72a174ac4a6a7f39da3065462 [sys.catalog]: This master's current role is: LEARNER
I20260806 04:58:05.636261 19975 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 6126c11fed1e4765becce7bd422ce904: attempting to promote NON_VOTER 50f00ca72a174ac4a6a7f39da3065462 to VOTER
I20260806 04:58:05.637038 19975 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6126c11fed1e4765becce7bd422ce904 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "6126c11fed1e4765becce7bd422ce904" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 40031 } } peers { permanent_uuid: "89e9df10c70d4ddaacd2a30fff7a3b2d" member_type: VOTER last_known_addr { host: "127.9.96.125" port: 39617 } attrs { promote: false } } peers { permanent_uuid: "50f00ca72a174ac4a6a7f39da3065462" member_type: VOTER last_known_addr { host: "127.9.96.124" port: 44477 } attrs { promote: false } }
I20260806 04:58:05.638967 20194 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 50f00ca72a174ac4a6a7f39da3065462 [term 1 LEARNER]: Refusing update from remote peer 6126c11fed1e4765becce7bd422ce904: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20260806 04:58:05.639022 19885 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 89e9df10c70d4ddaacd2a30fff7a3b2d [term 1 FOLLOWER]: Refusing update from remote peer 6126c11fed1e4765becce7bd422ce904: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20260806 04:58:05.639680 19802 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 6126c11fed1e4765becce7bd422ce904 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50f00ca72a174ac4a6a7f39da3065462" member_type: VOTER last_known_addr { host: "127.9.96.124" port: 44477 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260806 04:58:05.639926 19975 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 6126c11fed1e4765becce7bd422ce904 [LEADER]: Connected to new peer: Peer: permanent_uuid: "89e9df10c70d4ddaacd2a30fff7a3b2d" member_type: VOTER last_known_addr { host: "127.9.96.125" port: 39617 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260806 04:58:05.642287 19974 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 6126c11fed1e4765becce7bd422ce904 [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 50f00ca72a174ac4a6a7f39da3065462 (127.9.96.124) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "6126c11fed1e4765becce7bd422ce904" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 40031 } } peers { permanent_uuid: "89e9df10c70d4ddaacd2a30fff7a3b2d" member_type: VOTER last_known_addr { host: "127.9.96.125" port: 39617 } attrs { promote: false } } peers { permanent_uuid: "50f00ca72a174ac4a6a7f39da3065462" member_type: VOTER last_known_addr { host: "127.9.96.124" port: 44477 } attrs { promote: false } } }
I20260806 04:58:05.642833 19885 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 89e9df10c70d4ddaacd2a30fff7a3b2d [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 50f00ca72a174ac4a6a7f39da3065462 (127.9.96.124) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "6126c11fed1e4765becce7bd422ce904" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 40031 } } peers { permanent_uuid: "89e9df10c70d4ddaacd2a30fff7a3b2d" member_type: VOTER last_known_addr { host: "127.9.96.125" port: 39617 } attrs { promote: false } } peers { permanent_uuid: "50f00ca72a174ac4a6a7f39da3065462" member_type: VOTER last_known_addr { host: "127.9.96.124" port: 44477 } attrs { promote: false } } }
I20260806 04:58:05.646281 19974 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6126c11fed1e4765becce7bd422ce904 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "6126c11fed1e4765becce7bd422ce904" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "6126c11fed1e4765becce7bd422ce904" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 40031 } } peers { permanent_uuid: "89e9df10c70d4ddaacd2a30fff7a3b2d" member_type: VOTER last_known_addr { host: "127.9.96.125" port: 39617 } attrs { promote: false } } peers { permanent_uuid: "50f00ca72a174ac4a6a7f39da3065462" member_type: VOTER last_known_addr { host: "127.9.96.124" port: 44477 } attrs { promote: false } } }
I20260806 04:58:05.646620 19974 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6126c11fed1e4765becce7bd422ce904 [sys.catalog]: This master's current role is: LEADER
I20260806 04:58:05.646795 19802 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6126c11fed1e4765becce7bd422ce904 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "6126c11fed1e4765becce7bd422ce904" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "6126c11fed1e4765becce7bd422ce904" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 40031 } } peers { permanent_uuid: "89e9df10c70d4ddaacd2a30fff7a3b2d" member_type: VOTER last_known_addr { host: "127.9.96.125" port: 39617 } attrs { promote: false } } peers { permanent_uuid: "50f00ca72a174ac4a6a7f39da3065462" member_type: VOTER last_known_addr { host: "127.9.96.124" port: 44477 } attrs { promote: false } } }
I20260806 04:58:05.647126 19802 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6126c11fed1e4765becce7bd422ce904 [sys.catalog]: This master's current role is: LEADER
I20260806 04:58:05.647677 19802 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6126c11fed1e4765becce7bd422ce904 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "6126c11fed1e4765becce7bd422ce904" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "6126c11fed1e4765becce7bd422ce904" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 40031 } } peers { permanent_uuid: "89e9df10c70d4ddaacd2a30fff7a3b2d" member_type: VOTER last_known_addr { host: "127.9.96.125" port: 39617 } attrs { promote: false } } peers { permanent_uuid: "50f00ca72a174ac4a6a7f39da3065462" member_type: VOTER last_known_addr { host: "127.9.96.124" port: 44477 } attrs { promote: false } } }
I20260806 04:58:05.647975 19802 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6126c11fed1e4765becce7bd422ce904 [sys.catalog]: This master's current role is: LEADER
I20260806 04:58:05.648213 20261 sys_catalog.cc:455] T 00000000000000000000000000000000 P 89e9df10c70d4ddaacd2a30fff7a3b2d [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "6126c11fed1e4765becce7bd422ce904" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "6126c11fed1e4765becce7bd422ce904" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 40031 } } peers { permanent_uuid: "89e9df10c70d4ddaacd2a30fff7a3b2d" member_type: VOTER last_known_addr { host: "127.9.96.125" port: 39617 } attrs { promote: false } } peers { permanent_uuid: "50f00ca72a174ac4a6a7f39da3065462" member_type: VOTER last_known_addr { host: "127.9.96.124" port: 44477 } attrs { promote: false } } }
I20260806 04:58:05.648499 20261 sys_catalog.cc:458] T 00000000000000000000000000000000 P 89e9df10c70d4ddaacd2a30fff7a3b2d [sys.catalog]: This master's current role is: FOLLOWER
I20260806 04:58:05.648448 20194 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 50f00ca72a174ac4a6a7f39da3065462 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 50f00ca72a174ac4a6a7f39da3065462 (127.9.96.124) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "6126c11fed1e4765becce7bd422ce904" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 40031 } } peers { permanent_uuid: "89e9df10c70d4ddaacd2a30fff7a3b2d" member_type: VOTER last_known_addr { host: "127.9.96.125" port: 39617 } attrs { promote: false } } peers { permanent_uuid: "50f00ca72a174ac4a6a7f39da3065462" member_type: VOTER last_known_addr { host: "127.9.96.124" port: 44477 } attrs { promote: false } } }
I20260806 04:58:05.651859 20236 sys_catalog.cc:455] T 00000000000000000000000000000000 P 50f00ca72a174ac4a6a7f39da3065462 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "6126c11fed1e4765becce7bd422ce904" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "6126c11fed1e4765becce7bd422ce904" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 40031 } } peers { permanent_uuid: "89e9df10c70d4ddaacd2a30fff7a3b2d" member_type: VOTER last_known_addr { host: "127.9.96.125" port: 39617 } attrs { promote: false } } peers { permanent_uuid: "50f00ca72a174ac4a6a7f39da3065462" member_type: VOTER last_known_addr { host: "127.9.96.124" port: 44477 } attrs { promote: false } } }
I20260806 04:58:05.652138 20236 sys_catalog.cc:458] T 00000000000000000000000000000000 P 50f00ca72a174ac4a6a7f39da3065462 [sys.catalog]: This master's current role is: FOLLOWER
W20260806 04:58:05.681797 20161 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 04:58:05.682056 20161 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 04:58:05.682219 20161 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 04:58:05.689913 20161 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 04:58:05.690033 20161 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 04:58:05.690208 20161 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.96.65
I20260806 04:58:05.699767 20161 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.96.84:34171
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785992227050689-9601-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785992227050689-9601-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785992227050689-9601-0/krb5kdc/oryx_127.9.96.65.keytab
--principal=oryx/127.9.96.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.96.65:37807
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785992227050689-9601-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.96.65
--webserver_port=37575
--tserver_master_addrs=127.9.96.126:40031,127.9.96.125:39617,127.9.96.124:44477
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.96.65
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785992227050689-9601-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 03:56:12 UTC on ccycloud.quasar-guijbm.root.comops.site
build id 4569618
ASAN enabled
I20260806 04:58:05.700974 20161 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 04:58:05.702286 20161 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 04:58:05.715903 20270 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:58:05.716238 20273 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:58:05.716477 20269 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:58:05.716964 20161 server_base.cc:1034] running on GCE node
I20260806 04:58:05.717694 20161 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 04:58:05.718690 20161 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 04:58:05.719877 20161 hybrid_clock.cc:648] HybridClock initialized: now 1785992285719818 us; error 48 us; skew 500 ppm
I20260806 04:58:05.720142 20161 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 04:58:05 dist-test-slave-x4qh krb5kdc[18896](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992285, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.96.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 04:58:05.727630 20161 init.cc:377] Logged in from keytab as oryx/127.9.96.65@KRBTEST.COM (short username oryx)
I20260806 04:58:05.728999 20161 webserver.cc:460] Webserver started at http://127.9.96.65:37575/ using document root <none> and password file <none>
I20260806 04:58:05.729625 20161 fs_manager.cc:359] Metadata directory not provided
I20260806 04:58:05.729738 20161 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 04:58:05.734731 20161 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260806 04:58:05.737298 20286 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 04:58:05.738653 20161 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 04:58:05.738811 20161 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785992227050689-9601-0/minicluster-data/ts-0/data,/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785992227050689-9601-0/minicluster-data/ts-0/wal
uuid: "2259df3ac9ca47bfb61228d968d524fd"
format_stamp: "Formatted at 2026-08-06 04:58:02 on dist-test-slave-x4qh"
server_key: "d04f51fa2c3f6878369102bba77c9d82"
server_key_iv: "1e1993b305485b2d8c5822cb57678502"
server_key_version: "encryptionkey@0"
I20260806 04:58:05.739826 20161 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785992227050689-9601-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785992227050689-9601-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785992227050689-9601-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 04:58:05.763604 20161 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 04:58:05.767611 20161 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 04:58:05.767876 20161 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 04:58:05.768922 20161 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 04:58:05.770310 20161 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 04:58:05.770437 20161 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 04:58:05.770560 20161 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 04:58:05.770651 20161 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 04:58:05.806042 20161 rpc_server.cc:307] RPC server started. Bound to: 127.9.96.65:37807
I20260806 04:58:05.806126 20443 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.96.65:37807 every 8 connection(s)
I20260806 04:58:05.807526 20161 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785992227050689-9601-0/minicluster-data/ts-0/data/info.pb
I20260806 04:58:05.815080 9601 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu as pid 20161
I20260806 04:58:05.815456 9601 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu pid 19240
Aug 06 04:58:05 dist-test-slave-x4qh krb5kdc[18896](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992281, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.9.96.66@KRBTEST.COM
Aug 06 04:58:05 dist-test-slave-x4qh krb5kdc[18896](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992285, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.96.65@KRBTEST.COM for oryx/127.9.96.124@KRBTEST.COM
Aug 06 04:58:05 dist-test-slave-x4qh krb5kdc[18896](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992285, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.96.65@KRBTEST.COM for oryx/127.9.96.126@KRBTEST.COM
Aug 06 04:58:05 dist-test-slave-x4qh krb5kdc[18896](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992285, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.96.65@KRBTEST.COM for oryx/127.9.96.125@KRBTEST.COM
I20260806 04:58:05.847271 20461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:58:05.819534 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54536 (local address 127.9.96.66:41607)
0806 04:58:05.821041 (+ 1507us) server_negotiation.cc:206] Beginning negotiation
0806 04:58:05.821052 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 04:58:05.821087 (+ 35us) server_negotiation.cc:407] Connection header received
0806 04:58:05.821170 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:58:05.821181 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:58:05.821517 (+ 336us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:58:05.825803 (+ 4286us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:58:05.826116 (+ 313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:05.827938 (+ 1822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:05.830023 (+ 2085us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:05.830962 (+ 939us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:05.836447 (+ 5485us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:58:05.838639 (+ 2192us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:58:05.838653 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:58:05.838669 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:58:05.838709 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:58:05.842924 (+ 4215us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:58:05.843818 (+ 894us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:58:05.843828 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:58:05.843831 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:58:05.844050 (+ 219us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:58:05.844538 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:58:05.844543 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:58:05.844547 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:58:05.845096 (+ 549us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:58:05.846489 (+ 1393us) server_negotiation.cc:1035] Waiting for connection context
0806 04:58:05.846643 (+ 154us) server_negotiation.cc:299] Negotiation successful
0806 04:58:05.846910 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1297,"thread_start_us":2189,"threads_started":1}
I20260806 04:58:05.848832 19424 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 04:58:05.853034 20463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:58:05.822277 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.125:39617 (local address 127.9.96.65:45109)
0806 04:58:05.828811 (+ 6534us) negotiation.cc:107] Waiting for socket to connect
0806 04:58:05.828836 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0806 04:58:05.828986 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:58:05.829338 (+ 352us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:58:05.829344 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:58:05.829954 (+ 610us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:58:05.830294 (+ 340us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:58:05.830307 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:58:05.831788 (+ 1481us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:58:05.831793 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:58:05.833367 (+ 1574us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:58:05.833382 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:58:05.834639 (+ 1257us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:58:05.834646 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:58:05.834953 (+ 307us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:58:05.835730 (+ 777us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:58:05.835759 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:58:05.842161 (+ 6402us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 04:58:05.847586 (+ 5425us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:58:05.847596 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:58:05.847614 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:58:05.848257 (+ 643us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:58:05.848746 (+ 489us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:58:05.848754 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:58:05.848760 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:58:05.849063 (+ 303us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:58:05.849707 (+ 644us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 04:58:05.849727 (+ 20us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 04:58:05.850269 (+ 542us) client_negotiation.cc:769] Sending connection context
0806 04:58:05.850681 (+ 412us) client_negotiation.cc:240] Negotiation successful
0806 04:58:05.851362 (+ 681us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6404,"mutex_wait_us":5,"thread_start_us":6804,"threads_started":1}
I20260806 04:58:05.854138 20462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:58:05.820847 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.96.65:45109 (local address 127.9.96.125:39617)
0806 04:58:05.822373 (+ 1526us) server_negotiation.cc:206] Beginning negotiation
0806 04:58:05.822381 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 04:58:05.828934 (+ 6553us) server_negotiation.cc:407] Connection header received
0806 04:58:05.829124 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:58:05.829129 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:58:05.829212 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:58:05.829320 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:58:05.830476 (+ 1156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:05.831610 (+ 1134us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:05.833544 (+ 1934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:05.834230 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:05.834338 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:58:05.842373 (+ 8035us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:58:05.842380 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:58:05.842385 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:58:05.842434 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:58:05.847444 (+ 5010us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:58:05.848427 (+ 983us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:58:05.848433 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:58:05.848436 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:58:05.848593 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:58:05.849206 (+ 613us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:58:05.849213 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:58:05.849219 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:58:05.849563 (+ 344us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:58:05.849699 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0806 04:58:05.853553 (+ 3854us) server_negotiation.cc:299] Negotiation successful
0806 04:58:05.853783 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1353,"thread_start_us":1204,"threads_started":1}
I20260806 04:58:05.855574 20447 heartbeater.cc:347] Connected to a master server at 127.9.96.125:39617
I20260806 04:58:05.855917 20447 heartbeater.cc:464] Registering TS with master...
I20260806 04:58:05.856858 20447 heartbeater.cc:511] Master 127.9.96.125:39617 requested a full tablet report, sending...
I20260806 04:58:05.859992 19868 ts_manager.cc:194] Registered new tserver with Master: 2259df3ac9ca47bfb61228d968d524fd (127.9.96.65:37807)
I20260806 04:58:05.865262 20452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:58:05.811390 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.96.65:44643 (local address 127.9.96.126:40031)
0806 04:58:05.813295 (+ 1905us) server_negotiation.cc:206] Beginning negotiation
0806 04:58:05.813304 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 04:58:05.820039 (+ 6735us) server_negotiation.cc:407] Connection header received
0806 04:58:05.822981 (+ 2942us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:58:05.822990 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:58:05.823102 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:58:05.823196 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:58:05.831077 (+ 7881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:05.832300 (+ 1223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:05.835542 (+ 3242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:05.836227 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:05.836376 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:58:05.846934 (+ 10558us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:58:05.846942 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:58:05.846947 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:58:05.846996 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:58:05.861296 (+ 14300us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:58:05.862227 (+ 931us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:58:05.862233 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:58:05.862237 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:58:05.862426 (+ 189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:58:05.862977 (+ 551us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:58:05.862984 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:58:05.862990 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:58:05.863474 (+ 484us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:58:05.863631 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0806 04:58:05.864369 (+ 738us) server_negotiation.cc:299] Negotiation successful
0806 04:58:05.864689 (+ 320us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1678,"thread_start_us":3996,"threads_started":1}
I20260806 04:58:05.866667 20455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:58:05.811072 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:40031 (local address 127.9.96.65:44643)
0806 04:58:05.819935 (+ 8863us) negotiation.cc:107] Waiting for socket to connect
0806 04:58:05.819958 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0806 04:58:05.822594 (+ 2636us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:58:05.823915 (+ 1321us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:58:05.823930 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:58:05.824976 (+ 1046us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:58:05.826111 (+ 1135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:58:05.826143 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:58:05.832504 (+ 6361us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:58:05.832510 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:58:05.833994 (+ 1484us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:58:05.834008 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:58:05.836993 (+ 2985us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:58:05.837002 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:58:05.837180 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:58:05.837875 (+ 695us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:58:05.837897 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:58:05.841814 (+ 3917us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 04:58:05.861500 (+ 19686us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:58:05.861506 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:58:05.861510 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:58:05.862050 (+ 540us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:58:05.862606 (+ 556us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:58:05.862613 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:58:05.862617 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:58:05.862821 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:58:05.863758 (+ 937us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 04:58:05.863765 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 04:58:05.863987 (+ 222us) client_negotiation.cc:769] Sending connection context
0806 04:58:05.865537 (+ 1550us) client_negotiation.cc:240] Negotiation successful
0806 04:58:05.865946 (+ 409us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8716,"mutex_wait_us":106,"spinlock_wait_cycles":153344,"thread_start_us":8367,"threads_started":1}
I20260806 04:58:05.867223 20256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:58:05.811785 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.96.65:45863 (local address 127.9.96.124:44477)
0806 04:58:05.812044 (+ 259us) server_negotiation.cc:206] Beginning negotiation
0806 04:58:05.812053 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 04:58:05.819999 (+ 7946us) server_negotiation.cc:407] Connection header received
0806 04:58:05.824236 (+ 4237us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:58:05.824244 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:58:05.824347 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:58:05.824437 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:58:05.826778 (+ 2341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:05.828244 (+ 1466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:05.834169 (+ 5925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:05.834850 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:05.834950 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:58:05.847825 (+ 12875us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:58:05.847830 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:58:05.847833 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:58:05.847868 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:58:05.861266 (+ 13398us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:58:05.864332 (+ 3066us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:58:05.864338 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:58:05.864343 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:58:05.864574 (+ 231us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:58:05.865064 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:58:05.865071 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:58:05.865075 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:58:05.865458 (+ 383us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:58:05.865618 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0806 04:58:05.866554 (+ 936us) server_negotiation.cc:299] Negotiation successful
0806 04:58:05.866826 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260806 04:58:05.867638 20445 heartbeater.cc:347] Connected to a master server at 127.9.96.126:40031
I20260806 04:58:05.867779 20445 heartbeater.cc:464] Registering TS with master...
I20260806 04:58:05.868336 20445 heartbeater.cc:511] Master 127.9.96.126:40031 requested a full tablet report, sending...
I20260806 04:58:05.869184 20453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:58:05.811072 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.124:44477 (local address 127.9.96.65:45863)
0806 04:58:05.819789 (+ 8717us) negotiation.cc:107] Waiting for socket to connect
0806 04:58:05.819864 (+ 75us) client_negotiation.cc:174] Beginning negotiation
0806 04:58:05.824042 (+ 4178us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:58:05.824663 (+ 621us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:58:05.824668 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:58:05.825360 (+ 692us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:58:05.826571 (+ 1211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:58:05.826587 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:58:05.828533 (+ 1946us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:58:05.828541 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:58:05.830858 (+ 2317us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:58:05.830872 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:58:05.835267 (+ 4395us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:58:05.835277 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:58:05.836581 (+ 1304us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:58:05.837281 (+ 700us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:58:05.837304 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:58:05.842452 (+ 5148us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 04:58:05.863518 (+ 21066us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:58:05.863524 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:58:05.863529 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:58:05.864131 (+ 602us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:58:05.864759 (+ 628us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:58:05.864764 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:58:05.864769 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:58:05.864926 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:58:05.866024 (+ 1098us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 04:58:05.866028 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 04:58:05.866201 (+ 173us) client_negotiation.cc:769] Sending connection context
0806 04:58:05.868141 (+ 1940us) client_negotiation.cc:240] Negotiation successful
0806 04:58:05.868628 (+ 487us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8369,"mutex_wait_us":620,"thread_start_us":8822,"threads_started":1}
I20260806 04:58:05.870074 18960 ts_manager.cc:194] Re-registered known tserver with Master: 2259df3ac9ca47bfb61228d968d524fd (127.9.96.65:37807)
I20260806 04:58:05.870292 20444 heartbeater.cc:347] Connected to a master server at 127.9.96.124:44477
I20260806 04:58:05.870419 20444 heartbeater.cc:464] Registering TS with master...
I20260806 04:58:05.870819 20444 heartbeater.cc:511] Master 127.9.96.124:44477 requested a full tablet report, sending...
I20260806 04:58:05.871317 18960 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.9.96.65@KRBTEST.COM'} at 127.9.96.65:44643
I20260806 04:58:05.872839 20179 ts_manager.cc:194] Registered new tserver with Master: 2259df3ac9ca47bfb61228d968d524fd (127.9.96.65:37807)
I20260806 04:58:05.945657 9601 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu with pid 19240
I20260806 04:58:05.970033 9601 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
/tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785992227050689-9601-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785992227050689-9601-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785992227050689-9601-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785992227050689-9601-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.96.66:41607
--local_ip_for_outbound_sockets=127.9.96.66
--tserver_master_addrs=127.9.96.126:40031
--webserver_port=45523
--webserver_interface=127.9.96.66
--keytab_file=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785992227050689-9601-0/krb5kdc/oryx_127.9.96.66.keytab
--principal=oryx/127.9.96.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.96.84:34171
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.9.96.126:40031,127.9.96.125:39617
--tserver_master_addrs=127.9.96.126:40031,127.9.96.125:39617,127.9.96.124:44477 with env {KRB5CCNAME=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785992227050689-9601-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785992227050689-9601-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785992227050689-9601-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 04:58:06.314839 20466 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 04:58:06.315196 20466 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 04:58:06.315416 20466 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 04:58:06.323341 20466 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 04:58:06.323503 20466 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 04:58:06.323704 20466 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.96.66
I20260806 04:58:06.333204 20466 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.96.84:34171
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785992227050689-9601-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785992227050689-9601-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785992227050689-9601-0/krb5kdc/oryx_127.9.96.66.keytab
--principal=oryx/127.9.96.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.96.66:41607
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785992227050689-9601-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.96.66
--webserver_port=45523
--tserver_master_addrs=127.9.96.126:40031,127.9.96.125:39617,127.9.96.124:44477
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.96.66
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785992227050689-9601-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 03:56:12 UTC on ccycloud.quasar-guijbm.root.comops.site
build id 4569618
ASAN enabled
I20260806 04:58:06.334446 20466 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 04:58:06.335764 20466 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 04:58:06.349273 20473 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:58:06.349238 20476 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:58:06.349191 20484 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:58:06.350463 20466 server_base.cc:1034] running on GCE node
I20260806 04:58:06.350996 20466 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 04:58:06.351907 20466 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 04:58:06.353137 20466 hybrid_clock.cc:648] HybridClock initialized: now 1785992286353098 us; error 47 us; skew 500 ppm
I20260806 04:58:06.353463 20466 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 04:58:06 dist-test-slave-x4qh krb5kdc[18896](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992286, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.96.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 04:58:06.356631 20466 init.cc:377] Logged in from keytab as oryx/127.9.96.66@KRBTEST.COM (short username oryx)
I20260806 04:58:06.358076 20466 webserver.cc:460] Webserver started at http://127.9.96.66:45523/ using document root <none> and password file <none>
I20260806 04:58:06.358819 20466 fs_manager.cc:359] Metadata directory not provided
I20260806 04:58:06.358994 20466 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 04:58:06.364436 20466 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260806 04:58:06.367230 20498 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 04:58:06.368615 20466 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 04:58:06.368801 20466 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785992227050689-9601-0/minicluster-data/ts-1/data,/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785992227050689-9601-0/minicluster-data/ts-1/wal
uuid: "50ea117b4ffa45bc8456eb9ec24594b3"
format_stamp: "Formatted at 2026-08-06 04:58:03 on dist-test-slave-x4qh"
server_key: "6be8ccac887ffa4d995fbf54c2a43266"
server_key_iv: "def13310c91e1f0809786561febe5862"
server_key_version: "encryptionkey@0"
I20260806 04:58:06.369957 20466 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785992227050689-9601-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785992227050689-9601-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785992227050689-9601-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 04:58:06.381546 20466 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 04:58:06.385272 20466 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 04:58:06.385552 20466 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 04:58:06.386493 20466 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 04:58:06.387822 20466 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 04:58:06.387949 20466 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 04:58:06.388079 20466 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 04:58:06.388172 20466 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 04:58:06.421873 20466 rpc_server.cc:307] RPC server started. Bound to: 127.9.96.66:41607
I20260806 04:58:06.421960 20681 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.96.66:41607 every 8 connection(s)
I20260806 04:58:06.423332 20466 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785992227050689-9601-0/minicluster-data/ts-1/data/info.pb
I20260806 04:58:06.433055 9601 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu as pid 20466
I20260806 04:58:06.433382 9601 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu pid 19439
Aug 06 04:58:06 dist-test-slave-x4qh krb5kdc[18896](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992286, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.96.66@KRBTEST.COM for oryx/127.9.96.124@KRBTEST.COM
Aug 06 04:58:06 dist-test-slave-x4qh krb5kdc[18896](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992286, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.96.66@KRBTEST.COM for oryx/127.9.96.125@KRBTEST.COM
Aug 06 04:58:06 dist-test-slave-x4qh krb5kdc[18896](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992286, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.96.66@KRBTEST.COM for oryx/127.9.96.126@KRBTEST.COM
I20260806 04:58:06.463668 20695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:58:06.426437 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.96.66:50847 (local address 127.9.96.124:44477)
0806 04:58:06.433287 (+ 6850us) server_negotiation.cc:206] Beginning negotiation
0806 04:58:06.433293 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 04:58:06.440125 (+ 6832us) server_negotiation.cc:407] Connection header received
0806 04:58:06.442674 (+ 2549us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:58:06.442685 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:58:06.442803 (+ 118us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:58:06.442919 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:58:06.446037 (+ 3118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:06.447639 (+ 1602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:06.450482 (+ 2843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:06.450935 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:06.451065 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:58:06.456240 (+ 5175us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:58:06.456247 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:58:06.456252 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:58:06.456290 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:58:06.459597 (+ 3307us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:58:06.460728 (+ 1131us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:58:06.460733 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:58:06.460736 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:58:06.460895 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:58:06.461471 (+ 576us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:58:06.461476 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:58:06.461480 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:58:06.461819 (+ 339us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:58:06.461969 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0806 04:58:06.462999 (+ 1030us) server_negotiation.cc:299] Negotiation successful
0806 04:58:06.463234 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6726,"thread_start_us":6603,"threads_started":1}
I20260806 04:58:06.464879 20698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:58:06.426877 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.124:44477 (local address 127.9.96.66:50847)
0806 04:58:06.439983 (+ 13106us) negotiation.cc:107] Waiting for socket to connect
0806 04:58:06.440012 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0806 04:58:06.442292 (+ 2280us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:58:06.443516 (+ 1224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:58:06.443532 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:58:06.444776 (+ 1244us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:58:06.445766 (+ 990us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:58:06.445795 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:58:06.447878 (+ 2083us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:58:06.447884 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:58:06.450339 (+ 2455us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:58:06.450351 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:58:06.451441 (+ 1090us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:58:06.451451 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:58:06.451741 (+ 290us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:58:06.452523 (+ 782us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:58:06.452562 (+ 39us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:58:06.456015 (+ 3453us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 04:58:06.459788 (+ 3773us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:58:06.459797 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:58:06.459811 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:58:06.460588 (+ 777us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:58:06.461048 (+ 460us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:58:06.461052 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:58:06.461055 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:58:06.461329 (+ 274us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:58:06.462072 (+ 743us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 04:58:06.462080 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 04:58:06.462457 (+ 377us) client_negotiation.cc:769] Sending connection context
0806 04:58:06.462885 (+ 428us) client_negotiation.cc:240] Negotiation successful
0806 04:58:06.463400 (+ 515us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12878,"mutex_wait_us":79,"spinlock_wait_cycles":15104,"thread_start_us":13177,"threads_started":1}
I20260806 04:58:06.466703 20685 heartbeater.cc:347] Connected to a master server at 127.9.96.124:44477
I20260806 04:58:06.467059 20685 heartbeater.cc:464] Registering TS with master...
I20260806 04:58:06.467550 20692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:58:06.426228 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.96.66:55553 (local address 127.9.96.125:39617)
0806 04:58:06.445709 (+ 19481us) server_negotiation.cc:206] Beginning negotiation
0806 04:58:06.445718 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 04:58:06.445742 (+ 24us) server_negotiation.cc:407] Connection header received
0806 04:58:06.445809 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:58:06.445817 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:58:06.445908 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:58:06.446028 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:58:06.447464 (+ 1436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:06.448767 (+ 1303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:06.450967 (+ 2200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:06.451607 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:06.452346 (+ 739us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:58:06.457004 (+ 4658us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:58:06.457011 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:58:06.457016 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:58:06.457055 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:58:06.462196 (+ 5141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:58:06.464000 (+ 1804us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:58:06.464005 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:58:06.464009 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:58:06.464159 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:58:06.464674 (+ 515us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:58:06.464680 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:58:06.464685 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:58:06.465184 (+ 499us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:58:06.466748 (+ 1564us) server_negotiation.cc:1035] Waiting for connection context
0806 04:58:06.466908 (+ 160us) server_negotiation.cc:299] Negotiation successful
0806 04:58:06.467139 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19325,"thread_start_us":19940,"threads_started":1}
I20260806 04:58:06.467906 20696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:58:06.426872 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.125:39617 (local address 127.9.96.66:55553)
0806 04:58:06.439095 (+ 12223us) negotiation.cc:107] Waiting for socket to connect
0806 04:58:06.439143 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0806 04:58:06.442274 (+ 3131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:58:06.446360 (+ 4086us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:58:06.446368 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:58:06.446904 (+ 536us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:58:06.447218 (+ 314us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:58:06.447232 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:58:06.448938 (+ 1706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:58:06.448943 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:58:06.450803 (+ 1860us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:58:06.450822 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:58:06.451821 (+ 999us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:58:06.451827 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:58:06.452044 (+ 217us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:58:06.452646 (+ 602us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:58:06.452673 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:58:06.456751 (+ 4078us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 04:58:06.462344 (+ 5593us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:58:06.462349 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:58:06.462353 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:58:06.462950 (+ 597us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:58:06.464300 (+ 1350us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:58:06.464305 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:58:06.464308 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:58:06.464516 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:58:06.465417 (+ 901us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 04:58:06.465426 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 04:58:06.465588 (+ 162us) client_negotiation.cc:769] Sending connection context
0806 04:58:06.465746 (+ 158us) client_negotiation.cc:240] Negotiation successful
0806 04:58:06.466037 (+ 291us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11914,"mutex_wait_us":4,"thread_start_us":11449,"threads_started":1}
I20260806 04:58:06.468035 20685 heartbeater.cc:511] Master 127.9.96.124:44477 requested a full tablet report, sending...
I20260806 04:58:06.468843 20690 heartbeater.cc:347] Connected to a master server at 127.9.96.125:39617
I20260806 04:58:06.469005 20690 heartbeater.cc:464] Registering TS with master...
Aug 06 04:58:06 dist-test-slave-x4qh krb5kdc[18896](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992281, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.9.96.67@KRBTEST.COM
I20260806 04:58:06.469544 20690 heartbeater.cc:511] Master 127.9.96.125:39617 requested a full tablet report, sending...
I20260806 04:58:06.469949 20179 ts_manager.cc:194] Registered new tserver with Master: 50ea117b4ffa45bc8456eb9ec24594b3 (127.9.96.66:41607)
I20260806 04:58:06.470705 19868 ts_manager.cc:194] Registered new tserver with Master: 50ea117b4ffa45bc8456eb9ec24594b3 (127.9.96.66:41607)
I20260806 04:58:06.473099 20702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:58:06.448938 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:40031 (local address 127.9.96.66:55449)
0806 04:58:06.451364 (+ 2426us) negotiation.cc:107] Waiting for socket to connect
0806 04:58:06.451404 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0806 04:58:06.451506 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:58:06.452423 (+ 917us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:58:06.452428 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:58:06.454287 (+ 1859us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:58:06.454632 (+ 345us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:58:06.454647 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:58:06.456050 (+ 1403us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:58:06.456055 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:58:06.457680 (+ 1625us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:58:06.457697 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:58:06.458827 (+ 1130us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:58:06.458835 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:58:06.459033 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:58:06.459602 (+ 569us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:58:06.459629 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:58:06.463233 (+ 3604us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 04:58:06.469623 (+ 6390us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:58:06.469628 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:58:06.469631 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:58:06.470126 (+ 495us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:58:06.471360 (+ 1234us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:58:06.471365 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:58:06.471393 (+ 28us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:58:06.471536 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:58:06.472245 (+ 709us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 04:58:06.472249 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 04:58:06.472381 (+ 132us) client_negotiation.cc:769] Sending connection context
0806 04:58:06.472481 (+ 100us) client_negotiation.cc:240] Negotiation successful
0806 04:58:06.472791 (+ 310us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2295,"thread_start_us":2170,"threads_started":1}
I20260806 04:58:06.473320 20701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:58:06.439693 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.96.66:55449 (local address 127.9.96.126:40031)
0806 04:58:06.451927 (+ 12234us) server_negotiation.cc:206] Beginning negotiation
0806 04:58:06.451936 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 04:58:06.451957 (+ 21us) server_negotiation.cc:407] Connection header received
0806 04:58:06.452023 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:58:06.452028 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:58:06.452111 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:58:06.452216 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:58:06.454813 (+ 2597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:06.455879 (+ 1066us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:06.457874 (+ 1995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:06.458559 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:06.459319 (+ 760us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:58:06.463482 (+ 4163us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:58:06.463489 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:58:06.463494 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:58:06.463534 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:58:06.469465 (+ 5931us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:58:06.471058 (+ 1593us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:58:06.471063 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:58:06.471066 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:58:06.471228 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:58:06.471703 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:58:06.471709 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:58:06.471712 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:58:06.472105 (+ 393us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:58:06.472263 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0806 04:58:06.472687 (+ 424us) server_negotiation.cc:299] Negotiation successful
0806 04:58:06.472931 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12091,"thread_start_us":16205,"threads_started":1}
I20260806 04:58:06.474130 20688 heartbeater.cc:347] Connected to a master server at 127.9.96.126:40031
I20260806 04:58:06.474265 20688 heartbeater.cc:464] Registering TS with master...
I20260806 04:58:06.474709 20688 heartbeater.cc:511] Master 127.9.96.126:40031 requested a full tablet report, sending...
I20260806 04:58:06.475711 18960 ts_manager.cc:194] Re-registered known tserver with Master: 50ea117b4ffa45bc8456eb9ec24594b3 (127.9.96.66:41607)
I20260806 04:58:06.476848 18960 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.9.96.66@KRBTEST.COM'} at 127.9.96.66:55449
I20260806 04:58:06.477702 20700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:58:06.434671 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39336 (local address 127.9.96.67:38449)
0806 04:58:06.453369 (+ 18698us) server_negotiation.cc:206] Beginning negotiation
0806 04:58:06.453378 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 04:58:06.453405 (+ 27us) server_negotiation.cc:407] Connection header received
0806 04:58:06.453470 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:58:06.453477 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:58:06.453709 (+ 232us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:58:06.453878 (+ 169us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:58:06.458936 (+ 5058us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:06.461059 (+ 2123us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:06.462919 (+ 1860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:06.463786 (+ 867us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:06.463918 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:58:06.470526 (+ 6608us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:58:06.470540 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:58:06.470574 (+ 34us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:58:06.470616 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:58:06.474724 (+ 4108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:58:06.475560 (+ 836us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:58:06.475567 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:58:06.475569 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:58:06.475695 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:58:06.476181 (+ 486us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:58:06.476186 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:58:06.476190 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:58:06.476601 (+ 411us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:58:06.476901 (+ 300us) server_negotiation.cc:1035] Waiting for connection context
0806 04:58:06.477146 (+ 245us) server_negotiation.cc:299] Negotiation successful
0806 04:58:06.477379 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18500,"thread_start_us":19343,"threads_started":1}
I20260806 04:58:06.478996 19625 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 04:58:06.564433 9601 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu with pid 19439
I20260806 04:58:06.591702 9601 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
/tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785992227050689-9601-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785992227050689-9601-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785992227050689-9601-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785992227050689-9601-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.96.67:38449
--local_ip_for_outbound_sockets=127.9.96.67
--tserver_master_addrs=127.9.96.126:40031
--webserver_port=38151
--webserver_interface=127.9.96.67
--keytab_file=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785992227050689-9601-0/krb5kdc/oryx_127.9.96.67.keytab
--principal=oryx/127.9.96.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.96.84:34171
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.9.96.126:40031,127.9.96.125:39617
--tserver_master_addrs=127.9.96.126:40031,127.9.96.125:39617,127.9.96.124:44477 with env {KRB5CCNAME=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785992227050689-9601-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785992227050689-9601-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785992227050689-9601-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 04:58:06.874437 20445 heartbeater.cc:503] Master 127.9.96.126:40031 was elected leader, sending a full tablet report...
W20260806 04:58:06.925092 20704 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 04:58:06.925400 20704 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 04:58:06.925568 20704 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 04:58:06.932951 20704 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 04:58:06.933089 20704 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 04:58:06.933255 20704 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.96.67
I20260806 04:58:06.942725 20704 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.96.84:34171
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785992227050689-9601-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785992227050689-9601-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785992227050689-9601-0/krb5kdc/oryx_127.9.96.67.keytab
--principal=oryx/127.9.96.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.96.67:38449
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785992227050689-9601-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.96.67
--webserver_port=38151
--tserver_master_addrs=127.9.96.126:40031,127.9.96.125:39617,127.9.96.124:44477
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.96.67
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785992227050689-9601-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 03:56:12 UTC on ccycloud.quasar-guijbm.root.comops.site
build id 4569618
ASAN enabled
I20260806 04:58:06.944164 20704 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 04:58:06.945555 20704 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 04:58:06.959319 20719 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:58:06.959437 20717 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:58:06.959400 20727 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:58:06.964833 20704 server_base.cc:1034] running on GCE node
I20260806 04:58:06.965481 20704 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 04:58:06.966504 20704 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 04:58:06.967723 20704 hybrid_clock.cc:648] HybridClock initialized: now 1785992286967685 us; error 32 us; skew 500 ppm
I20260806 04:58:06.968067 20704 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 04:58:06 dist-test-slave-x4qh krb5kdc[18896](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992286, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.96.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 04:58:06.971446 20704 init.cc:377] Logged in from keytab as oryx/127.9.96.67@KRBTEST.COM (short username oryx)
I20260806 04:58:06.972958 20704 webserver.cc:460] Webserver started at http://127.9.96.67:38151/ using document root <none> and password file <none>
I20260806 04:58:06.973678 20704 fs_manager.cc:359] Metadata directory not provided
I20260806 04:58:06.973871 20704 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 04:58:06.979348 20704 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260806 04:58:06.982102 20736 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 04:58:06.983345 20704 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260806 04:58:06.983564 20704 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785992227050689-9601-0/minicluster-data/ts-2/data,/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785992227050689-9601-0/minicluster-data/ts-2/wal
uuid: "aab3b242cd3e451e958340a207168137"
format_stamp: "Formatted at 2026-08-06 04:58:03 on dist-test-slave-x4qh"
server_key: "72dd9f89eb8120ef25689e7681aea634"
server_key_iv: "44939536f15b0f4e2aaaf9d56ce948a9"
server_key_version: "encryptionkey@0"
I20260806 04:58:06.984602 20704 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785992227050689-9601-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785992227050689-9601-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785992227050689-9601-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 04:58:07.009274 20704 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 04:58:07.013635 20704 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 04:58:07.014091 20704 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 04:58:07.015141 20704 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 04:58:07.016605 20704 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 04:58:07.016731 20704 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 04:58:07.016856 20704 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 04:58:07.016955 20704 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 04:58:07.048839 20704 rpc_server.cc:307] RPC server started. Bound to: 127.9.96.67:38449
I20260806 04:58:07.048892 20899 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.96.67:38449 every 8 connection(s)
I20260806 04:58:07.050539 20704 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785992227050689-9601-0/minicluster-data/ts-2/data/info.pb
I20260806 04:58:07.055471 9601 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu as pid 20704
Aug 06 04:58:07 dist-test-slave-x4qh krb5kdc[18896](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992286, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.96.67@KRBTEST.COM for oryx/127.9.96.126@KRBTEST.COM
Aug 06 04:58:07 dist-test-slave-x4qh krb5kdc[18896](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992286, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.96.67@KRBTEST.COM for oryx/127.9.96.124@KRBTEST.COM
I20260806 04:58:07.082845 20904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:58:07.053506 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.96.67:45747 (local address 127.9.96.126:40031)
0806 04:58:07.054009 (+ 503us) server_negotiation.cc:206] Beginning negotiation
0806 04:58:07.054025 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 04:58:07.055811 (+ 1786us) server_negotiation.cc:407] Connection header received
0806 04:58:07.059637 (+ 3826us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:58:07.059647 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:58:07.059772 (+ 125us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:58:07.059874 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:58:07.062988 (+ 3114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:07.064580 (+ 1592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:07.067900 (+ 3320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:07.068780 (+ 880us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:07.068897 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:58:07.074560 (+ 5663us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:58:07.074569 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:58:07.074576 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:58:07.074636 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:58:07.079028 (+ 4392us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:58:07.080065 (+ 1037us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:58:07.080071 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:58:07.080074 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:58:07.080253 (+ 179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:58:07.080799 (+ 546us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:58:07.080805 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:58:07.080808 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:58:07.081121 (+ 313us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:58:07.081262 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0806 04:58:07.082243 (+ 981us) server_negotiation.cc:299] Negotiation successful
0806 04:58:07.082464 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":343,"thread_start_us":420,"threads_started":1}
I20260806 04:58:07.084416 20905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:58:07.053821 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.96.67:48501 (local address 127.9.96.124:44477)
0806 04:58:07.054561 (+ 740us) server_negotiation.cc:206] Beginning negotiation
0806 04:58:07.054569 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 04:58:07.060486 (+ 5917us) server_negotiation.cc:407] Connection header received
0806 04:58:07.060792 (+ 306us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:58:07.060800 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:58:07.060902 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:58:07.060998 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:58:07.062987 (+ 1989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:07.064399 (+ 1412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:07.067844 (+ 3445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:07.068540 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:07.068634 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:58:07.075594 (+ 6960us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:58:07.075604 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:58:07.075611 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:58:07.075665 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:58:07.080412 (+ 4747us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:58:07.081471 (+ 1059us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:58:07.081479 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:58:07.081485 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:58:07.081658 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:58:07.082212 (+ 554us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:58:07.082233 (+ 21us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:58:07.082238 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:58:07.082651 (+ 413us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:58:07.082818 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0806 04:58:07.083536 (+ 718us) server_negotiation.cc:299] Negotiation successful
0806 04:58:07.083829 (+ 293us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":601,"thread_start_us":488,"threads_started":1}
I20260806 04:58:07.085916 20910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:58:07.054162 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:40031 (local address 127.9.96.67:45747)
0806 04:58:07.055296 (+ 1134us) negotiation.cc:107] Waiting for socket to connect
0806 04:58:07.055387 (+ 91us) client_negotiation.cc:174] Beginning negotiation
0806 04:58:07.059133 (+ 3746us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:58:07.060426 (+ 1293us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:58:07.060437 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:58:07.061638 (+ 1201us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:58:07.062734 (+ 1096us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:58:07.062763 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:58:07.064780 (+ 2017us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:58:07.064787 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:58:07.067702 (+ 2915us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:58:07.067722 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:58:07.069274 (+ 1552us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:58:07.069283 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:58:07.069563 (+ 280us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:58:07.070483 (+ 920us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:58:07.070528 (+ 45us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:58:07.074239 (+ 3711us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 04:58:07.079242 (+ 5003us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:58:07.079252 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:58:07.079274 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:58:07.079915 (+ 641us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:58:07.080410 (+ 495us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:58:07.080414 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:58:07.080418 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:58:07.080647 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:58:07.081348 (+ 701us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 04:58:07.081354 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 04:58:07.081807 (+ 453us) client_negotiation.cc:769] Sending connection context
0806 04:58:07.083331 (+ 1524us) client_negotiation.cc:240] Negotiation successful
0806 04:58:07.083922 (+ 591us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":778,"mutex_wait_us":142,"spinlock_wait_cycles":7552,"thread_start_us":5563,"threads_started":1}
I20260806 04:58:07.086036 20909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:58:07.054162 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.124:44477 (local address 127.9.96.67:48501)
0806 04:58:07.060306 (+ 6144us) negotiation.cc:107] Waiting for socket to connect
0806 04:58:07.060383 (+ 77us) client_negotiation.cc:174] Beginning negotiation
0806 04:58:07.060610 (+ 227us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:58:07.061280 (+ 670us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:58:07.061286 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:58:07.062144 (+ 858us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:58:07.062749 (+ 605us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:58:07.062770 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:58:07.064701 (+ 1931us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:58:07.064710 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:58:07.067676 (+ 2966us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:58:07.067690 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:58:07.069039 (+ 1349us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:58:07.069047 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:58:07.069414 (+ 367us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:58:07.070437 (+ 1023us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:58:07.070518 (+ 81us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:58:07.075280 (+ 4762us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 04:58:07.080604 (+ 5324us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:58:07.080612 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:58:07.080619 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:58:07.081300 (+ 681us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:58:07.081826 (+ 526us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:58:07.081834 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:58:07.081840 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:58:07.082063 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:58:07.082920 (+ 857us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 04:58:07.082928 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 04:58:07.083122 (+ 194us) client_negotiation.cc:769] Sending connection context
0806 04:58:07.084899 (+ 1777us) client_negotiation.cc:240] Negotiation successful
0806 04:58:07.085327 (+ 428us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5925,"mutex_wait_us":348,"spinlock_wait_cycles":36224,"thread_start_us":433,"threads_started":1}
Aug 06 04:58:07 dist-test-slave-x4qh krb5kdc[18896](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992286, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.96.67@KRBTEST.COM for oryx/127.9.96.125@KRBTEST.COM
I20260806 04:58:07.088089 20900 heartbeater.cc:347] Connected to a master server at 127.9.96.124:44477
I20260806 04:58:07.088145 20901 heartbeater.cc:347] Connected to a master server at 127.9.96.126:40031
I20260806 04:58:07.088454 20901 heartbeater.cc:464] Registering TS with master...
I20260806 04:58:07.088466 20900 heartbeater.cc:464] Registering TS with master...
I20260806 04:58:07.089485 20901 heartbeater.cc:511] Master 127.9.96.126:40031 requested a full tablet report, sending...
I20260806 04:58:07.090809 20900 heartbeater.cc:511] Master 127.9.96.124:44477 requested a full tablet report, sending...
I20260806 04:58:07.091303 18960 ts_manager.cc:194] Re-registered known tserver with Master: aab3b242cd3e451e958340a207168137 (127.9.96.67:38449)
I20260806 04:58:07.091950 20179 ts_manager.cc:194] Registered new tserver with Master: aab3b242cd3e451e958340a207168137 (127.9.96.67:38449)
I20260806 04:58:07.092471 18960 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.9.96.67@KRBTEST.COM'} at 127.9.96.67:45747
I20260806 04:58:07.095721 20918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:58:07.064684 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.125:39617 (local address 127.9.96.67:46315)
0806 04:58:07.077932 (+ 13248us) negotiation.cc:107] Waiting for socket to connect
0806 04:58:07.077952 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0806 04:58:07.078074 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:58:07.078902 (+ 828us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:58:07.078907 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:58:07.079580 (+ 673us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:58:07.079940 (+ 360us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:58:07.079953 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:58:07.081338 (+ 1385us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:58:07.081343 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:58:07.082853 (+ 1510us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:58:07.082870 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:58:07.084044 (+ 1174us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:58:07.084050 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:58:07.084244 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:58:07.084855 (+ 611us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:58:07.084878 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:58:07.088055 (+ 3177us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 04:58:07.092398 (+ 4343us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:58:07.092403 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:58:07.092407 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:58:07.093000 (+ 593us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:58:07.093465 (+ 465us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:58:07.093470 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:58:07.093474 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:58:07.093685 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:58:07.094474 (+ 789us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 04:58:07.094481 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 04:58:07.094667 (+ 186us) client_negotiation.cc:769] Sending connection context
0806 04:58:07.094813 (+ 146us) client_negotiation.cc:240] Negotiation successful
0806 04:58:07.095197 (+ 384us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13131,"mutex_wait_us":3,"thread_start_us":13492,"threads_started":1}
I20260806 04:58:07.097059 20916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:58:07.060485 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.96.67:46315 (local address 127.9.96.125:39617)
0806 04:58:07.077523 (+ 17038us) server_negotiation.cc:206] Beginning negotiation
0806 04:58:07.077529 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 04:58:07.078430 (+ 901us) server_negotiation.cc:407] Connection header received
0806 04:58:07.078502 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:58:07.078508 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:58:07.078592 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:58:07.078701 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:58:07.080114 (+ 1413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:07.081168 (+ 1054us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:07.083033 (+ 1865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:07.083668 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:07.083797 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:58:07.088318 (+ 4521us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:58:07.088324 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:58:07.088328 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:58:07.088383 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:58:07.092234 (+ 3851us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:58:07.093143 (+ 909us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:58:07.093148 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:58:07.093152 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:58:07.093323 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:58:07.093822 (+ 499us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:58:07.093827 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:58:07.093833 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:58:07.094242 (+ 409us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:58:07.094405 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0806 04:58:07.096105 (+ 1700us) server_negotiation.cc:299] Negotiation successful
0806 04:58:07.096488 (+ 383us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16905,"thread_start_us":17834,"threads_started":1}
I20260806 04:58:07.097929 20902 heartbeater.cc:347] Connected to a master server at 127.9.96.125:39617
I20260806 04:58:07.098073 20902 heartbeater.cc:464] Registering TS with master...
I20260806 04:58:07.098420 20902 heartbeater.cc:511] Master 127.9.96.125:39617 requested a full tablet report, sending...
I20260806 04:58:07.099556 19868 ts_manager.cc:194] Registered new tserver with Master: aab3b242cd3e451e958340a207168137 (127.9.96.67:38449)
I20260806 04:58:07.112592 20904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:58:07.092446 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48752 (local address 127.9.96.126:40031)
0806 04:58:07.093397 (+ 951us) server_negotiation.cc:206] Beginning negotiation
0806 04:58:07.093404 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 04:58:07.093420 (+ 16us) server_negotiation.cc:407] Connection header received
0806 04:58:07.093576 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:58:07.093581 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:58:07.093669 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:58:07.093737 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:58:07.096216 (+ 2479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:07.097042 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:07.100209 (+ 3167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:07.100903 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:07.101036 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:58:07.105006 (+ 3970us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:58:07.105014 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:58:07.105018 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:58:07.105062 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:58:07.108715 (+ 3653us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:58:07.109910 (+ 1195us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:58:07.109916 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:58:07.109921 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:58:07.110084 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:58:07.110562 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:58:07.110568 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:58:07.110571 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:58:07.110950 (+ 379us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:58:07.111202 (+ 252us) server_negotiation.cc:1035] Waiting for connection context
0806 04:58:07.111992 (+ 790us) server_negotiation.cc:299] Negotiation successful
0806 04:58:07.112246 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":830}
I20260806 04:58:07.114737 20916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:58:07.094026 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53026 (local address 127.9.96.125:39617)
0806 04:58:07.097368 (+ 3342us) server_negotiation.cc:206] Beginning negotiation
0806 04:58:07.097376 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 04:58:07.097400 (+ 24us) server_negotiation.cc:407] Connection header received
0806 04:58:07.097461 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:58:07.097469 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:58:07.097619 (+ 150us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:58:07.097747 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:58:07.100657 (+ 2910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:07.101561 (+ 904us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:07.103629 (+ 2068us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:07.104356 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:07.104503 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:58:07.107707 (+ 3204us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:58:07.107715 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:58:07.107722 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:58:07.107761 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:58:07.111015 (+ 3254us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:58:07.111707 (+ 692us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:58:07.111712 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:58:07.111715 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:58:07.111884 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:58:07.112484 (+ 600us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:58:07.112488 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:58:07.112491 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:58:07.113123 (+ 632us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:58:07.113396 (+ 273us) server_negotiation.cc:1035] Waiting for connection context
0806 04:58:07.113712 (+ 316us) server_negotiation.cc:299] Negotiation successful
0806 04:58:07.114505 (+ 793us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3210,"thread_start_us":6299,"threads_started":1}
I20260806 04:58:07.118034 20905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:58:07.101399 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55184 (local address 127.9.96.124:44477)
0806 04:58:07.101568 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0806 04:58:07.101574 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 04:58:07.101862 (+ 288us) server_negotiation.cc:407] Connection header received
0806 04:58:07.102282 (+ 420us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:58:07.102287 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:58:07.102371 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:58:07.102476 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:58:07.104528 (+ 2052us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:07.105415 (+ 887us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:07.107503 (+ 2088us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:07.108036 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:07.108173 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:58:07.111117 (+ 2944us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:58:07.111123 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:58:07.111128 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:58:07.111167 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:58:07.115053 (+ 3886us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:58:07.115935 (+ 882us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:58:07.115942 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:58:07.115945 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:58:07.116116 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:58:07.116545 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:58:07.116568 (+ 23us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:58:07.116572 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:58:07.116904 (+ 332us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:58:07.117161 (+ 257us) server_negotiation.cc:1035] Waiting for connection context
0806 04:58:07.117487 (+ 326us) server_negotiation.cc:299] Negotiation successful
0806 04:58:07.117729 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260806 04:58:07.117533 18960 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48752:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 04:58:07.125751 18960 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 04:58:07.155093 20945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:58:07.142202 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56358 (local address 127.9.96.65:37807)
0806 04:58:07.142782 (+ 580us) server_negotiation.cc:206] Beginning negotiation
0806 04:58:07.142791 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 04:58:07.143946 (+ 1155us) server_negotiation.cc:407] Connection header received
0806 04:58:07.144180 (+ 234us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:58:07.144189 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:58:07.144418 (+ 229us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:58:07.144690 (+ 272us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 04:58:07.145842 (+ 1152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:07.147620 (+ 1778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:07.151042 (+ 3422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:07.153078 (+ 2036us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:07.153361 (+ 283us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:58:07.153627 (+ 266us) server_negotiation.cc:1035] Waiting for connection context
0806 04:58:07.154437 (+ 810us) server_negotiation.cc:299] Negotiation successful
0806 04:58:07.154668 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":415,"thread_start_us":306,"threads_started":1}
I20260806 04:58:07.156352 20942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:58:07.141767 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.65:37807 (local address 127.0.0.1:56358)
0806 04:58:07.143826 (+ 2059us) negotiation.cc:107] Waiting for socket to connect
0806 04:58:07.143841 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 04:58:07.143991 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:58:07.145330 (+ 1339us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:58:07.145334 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:58:07.145362 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 04:58:07.145680 (+ 318us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:58:07.145689 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:58:07.147841 (+ 2152us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:58:07.147854 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:58:07.150833 (+ 2979us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:58:07.150853 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:58:07.153874 (+ 3021us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:58:07.153883 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:58:07.154153 (+ 270us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:58:07.154204 (+ 51us) client_negotiation.cc:769] Sending connection context
0806 04:58:07.155456 (+ 1252us) client_negotiation.cc:240] Negotiation successful
0806 04:58:07.155761 (+ 305us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1931,"mutex_wait_us":60,"thread_start_us":1784,"threads_started":1}
I20260806 04:58:07.160481 20946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:58:07.143934 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.66:41607 (local address 127.0.0.1:54552)
0806 04:58:07.146103 (+ 2169us) negotiation.cc:107] Waiting for socket to connect
0806 04:58:07.146119 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 04:58:07.146231 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:58:07.147271 (+ 1040us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:58:07.147279 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:58:07.147314 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 04:58:07.147896 (+ 582us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:58:07.147912 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:58:07.151727 (+ 3815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:58:07.151735 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:58:07.155765 (+ 4030us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:58:07.155779 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:58:07.159941 (+ 4162us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:58:07.159946 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:58:07.160061 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:58:07.160083 (+ 22us) client_negotiation.cc:769] Sending connection context
0806 04:58:07.160146 (+ 63us) client_negotiation.cc:240] Negotiation successful
0806 04:58:07.160242 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2039,"mutex_wait_us":3,"thread_start_us":1902,"threads_started":1}
I20260806 04:58:07.161229 20947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:58:07.144256 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54552 (local address 127.9.96.66:41607)
0806 04:58:07.144831 (+ 575us) server_negotiation.cc:206] Beginning negotiation
0806 04:58:07.144840 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 04:58:07.146368 (+ 1528us) server_negotiation.cc:407] Connection header received
0806 04:58:07.146493 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:58:07.146504 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:58:07.146800 (+ 296us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:58:07.147013 (+ 213us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 04:58:07.148154 (+ 1141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:07.151483 (+ 3329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:07.156012 (+ 4529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:07.159301 (+ 3289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:07.159548 (+ 247us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:58:07.160719 (+ 1171us) server_negotiation.cc:1035] Waiting for connection context
0806 04:58:07.160800 (+ 81us) server_negotiation.cc:299] Negotiation successful
0806 04:58:07.160967 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":425,"thread_start_us":830,"threads_started":1}
I20260806 04:58:07.163348 20348 tablet_service.cc:1467] Processing CreateTablet for tablet 0b9b038b47b24c7a80b020b1225b55e9 (DEFAULT_TABLE table=test-table [id=32ceabd633b947b6be332bd29e1530b8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 04:58:07.164312 20574 tablet_service.cc:1467] Processing CreateTablet for tablet 0b9b038b47b24c7a80b020b1225b55e9 (DEFAULT_TABLE table=test-table [id=32ceabd633b947b6be332bd29e1530b8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 04:58:07.164853 20348 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0b9b038b47b24c7a80b020b1225b55e9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 04:58:07.166224 20574 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0b9b038b47b24c7a80b020b1225b55e9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 04:58:07.167887 20942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:58:07.146169 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.67:38449 (local address 127.0.0.1:39344)
0806 04:58:07.156805 (+ 10636us) negotiation.cc:107] Waiting for socket to connect
0806 04:58:07.156828 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0806 04:58:07.156987 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:58:07.160845 (+ 3858us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:58:07.160850 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:58:07.160876 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 04:58:07.161031 (+ 155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:58:07.161040 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:58:07.163040 (+ 2000us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:58:07.163045 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:58:07.165015 (+ 1970us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:58:07.165030 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:58:07.167078 (+ 2048us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:58:07.167084 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:58:07.167283 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:58:07.167325 (+ 42us) client_negotiation.cc:769] Sending connection context
0806 04:58:07.167449 (+ 124us) client_negotiation.cc:240] Negotiation successful
0806 04:58:07.167588 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10519,"thread_start_us":13420,"threads_started":1}
I20260806 04:58:07.168071 20949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:58:07.146436 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39344 (local address 127.9.96.67:38449)
0806 04:58:07.159738 (+ 13302us) server_negotiation.cc:206] Beginning negotiation
0806 04:58:07.159748 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 04:58:07.159781 (+ 33us) server_negotiation.cc:407] Connection header received
0806 04:58:07.159896 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:58:07.159904 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:58:07.160134 (+ 230us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:58:07.160330 (+ 196us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 04:58:07.161245 (+ 915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:07.162886 (+ 1641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:07.165253 (+ 2367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:07.166896 (+ 1643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:07.167040 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:58:07.167239 (+ 199us) server_negotiation.cc:1035] Waiting for connection context
0806 04:58:07.167525 (+ 286us) server_negotiation.cc:299] Negotiation successful
0806 04:58:07.167740 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13107,"thread_start_us":14154,"threads_started":1}
I20260806 04:58:07.171896 20804 tablet_service.cc:1467] Processing CreateTablet for tablet 0b9b038b47b24c7a80b020b1225b55e9 (DEFAULT_TABLE table=test-table [id=32ceabd633b947b6be332bd29e1530b8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 04:58:07.173322 20804 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0b9b038b47b24c7a80b020b1225b55e9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 04:58:07.178292 20951 tablet_bootstrap.cc:492] T 0b9b038b47b24c7a80b020b1225b55e9 P 2259df3ac9ca47bfb61228d968d524fd: Bootstrap starting.
I20260806 04:58:07.181540 20951 tablet_bootstrap.cc:654] T 0b9b038b47b24c7a80b020b1225b55e9 P 2259df3ac9ca47bfb61228d968d524fd: Neither blocks nor log segments found. Creating new log.
I20260806 04:58:07.182631 20952 tablet_bootstrap.cc:492] T 0b9b038b47b24c7a80b020b1225b55e9 P 50ea117b4ffa45bc8456eb9ec24594b3: Bootstrap starting.
I20260806 04:58:07.182736 20951 log.cc:824] T 0b9b038b47b24c7a80b020b1225b55e9 P 2259df3ac9ca47bfb61228d968d524fd: Log is configured to *not* fsync() on all Append() calls
I20260806 04:58:07.186241 20951 tablet_bootstrap.cc:492] T 0b9b038b47b24c7a80b020b1225b55e9 P 2259df3ac9ca47bfb61228d968d524fd: No bootstrap required, opened a new log
I20260806 04:58:07.186313 20952 tablet_bootstrap.cc:654] T 0b9b038b47b24c7a80b020b1225b55e9 P 50ea117b4ffa45bc8456eb9ec24594b3: Neither blocks nor log segments found. Creating new log.
I20260806 04:58:07.186539 20951 ts_tablet_manager.cc:1397] T 0b9b038b47b24c7a80b020b1225b55e9 P 2259df3ac9ca47bfb61228d968d524fd: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.006s
I20260806 04:58:07.187753 20952 log.cc:824] T 0b9b038b47b24c7a80b020b1225b55e9 P 50ea117b4ffa45bc8456eb9ec24594b3: Log is configured to *not* fsync() on all Append() calls
I20260806 04:58:07.187963 20954 tablet_bootstrap.cc:492] T 0b9b038b47b24c7a80b020b1225b55e9 P aab3b242cd3e451e958340a207168137: Bootstrap starting.
I20260806 04:58:07.190919 20954 tablet_bootstrap.cc:654] T 0b9b038b47b24c7a80b020b1225b55e9 P aab3b242cd3e451e958340a207168137: Neither blocks nor log segments found. Creating new log.
I20260806 04:58:07.192210 20954 log.cc:824] T 0b9b038b47b24c7a80b020b1225b55e9 P aab3b242cd3e451e958340a207168137: Log is configured to *not* fsync() on all Append() calls
I20260806 04:58:07.195247 20954 tablet_bootstrap.cc:492] T 0b9b038b47b24c7a80b020b1225b55e9 P aab3b242cd3e451e958340a207168137: No bootstrap required, opened a new log
I20260806 04:58:07.195564 20954 ts_tablet_manager.cc:1397] T 0b9b038b47b24c7a80b020b1225b55e9 P aab3b242cd3e451e958340a207168137: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.002s
I20260806 04:58:07.196803 20952 tablet_bootstrap.cc:492] T 0b9b038b47b24c7a80b020b1225b55e9 P 50ea117b4ffa45bc8456eb9ec24594b3: No bootstrap required, opened a new log
I20260806 04:58:07.197070 20952 ts_tablet_manager.cc:1397] T 0b9b038b47b24c7a80b020b1225b55e9 P 50ea117b4ffa45bc8456eb9ec24594b3: Time spent bootstrapping tablet: real 0.015s user 0.000s sys 0.011s
I20260806 04:58:07.198403 20951 raft_consensus.cc:348] T 0b9b038b47b24c7a80b020b1225b55e9 P 2259df3ac9ca47bfb61228d968d524fd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2259df3ac9ca47bfb61228d968d524fd" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 37807 } } peers { permanent_uuid: "50ea117b4ffa45bc8456eb9ec24594b3" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 41607 } } peers { permanent_uuid: "aab3b242cd3e451e958340a207168137" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 38449 } }
I20260806 04:58:07.198817 20951 raft_consensus.cc:374] T 0b9b038b47b24c7a80b020b1225b55e9 P 2259df3ac9ca47bfb61228d968d524fd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 04:58:07.198916 20951 raft_consensus.cc:729] T 0b9b038b47b24c7a80b020b1225b55e9 P 2259df3ac9ca47bfb61228d968d524fd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2259df3ac9ca47bfb61228d968d524fd, State: Initialized, Role: FOLLOWER
I20260806 04:58:07.199569 20951 consensus_queue.cc:260] T 0b9b038b47b24c7a80b020b1225b55e9 P 2259df3ac9ca47bfb61228d968d524fd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2259df3ac9ca47bfb61228d968d524fd" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 37807 } } peers { permanent_uuid: "50ea117b4ffa45bc8456eb9ec24594b3" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 41607 } } peers { permanent_uuid: "aab3b242cd3e451e958340a207168137" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 38449 } }
I20260806 04:58:07.201257 20951 ts_tablet_manager.cc:1428] T 0b9b038b47b24c7a80b020b1225b55e9 P 2259df3ac9ca47bfb61228d968d524fd: Time spent starting tablet: real 0.015s user 0.014s sys 0.001s
I20260806 04:58:07.206527 20952 raft_consensus.cc:348] T 0b9b038b47b24c7a80b020b1225b55e9 P 50ea117b4ffa45bc8456eb9ec24594b3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2259df3ac9ca47bfb61228d968d524fd" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 37807 } } peers { permanent_uuid: "50ea117b4ffa45bc8456eb9ec24594b3" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 41607 } } peers { permanent_uuid: "aab3b242cd3e451e958340a207168137" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 38449 } }
I20260806 04:58:07.206862 20952 raft_consensus.cc:374] T 0b9b038b47b24c7a80b020b1225b55e9 P 50ea117b4ffa45bc8456eb9ec24594b3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 04:58:07.207000 20952 raft_consensus.cc:729] T 0b9b038b47b24c7a80b020b1225b55e9 P 50ea117b4ffa45bc8456eb9ec24594b3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 50ea117b4ffa45bc8456eb9ec24594b3, State: Initialized, Role: FOLLOWER
I20260806 04:58:07.207643 20952 consensus_queue.cc:260] T 0b9b038b47b24c7a80b020b1225b55e9 P 50ea117b4ffa45bc8456eb9ec24594b3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2259df3ac9ca47bfb61228d968d524fd" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 37807 } } peers { permanent_uuid: "50ea117b4ffa45bc8456eb9ec24594b3" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 41607 } } peers { permanent_uuid: "aab3b242cd3e451e958340a207168137" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 38449 } }
I20260806 04:58:07.207669 20954 raft_consensus.cc:348] T 0b9b038b47b24c7a80b020b1225b55e9 P aab3b242cd3e451e958340a207168137 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2259df3ac9ca47bfb61228d968d524fd" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 37807 } } peers { permanent_uuid: "50ea117b4ffa45bc8456eb9ec24594b3" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 41607 } } peers { permanent_uuid: "aab3b242cd3e451e958340a207168137" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 38449 } }
I20260806 04:58:07.208122 20954 raft_consensus.cc:374] T 0b9b038b47b24c7a80b020b1225b55e9 P aab3b242cd3e451e958340a207168137 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 04:58:07.208256 20954 raft_consensus.cc:729] T 0b9b038b47b24c7a80b020b1225b55e9 P aab3b242cd3e451e958340a207168137 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aab3b242cd3e451e958340a207168137, State: Initialized, Role: FOLLOWER
I20260806 04:58:07.208911 20954 consensus_queue.cc:260] T 0b9b038b47b24c7a80b020b1225b55e9 P aab3b242cd3e451e958340a207168137 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2259df3ac9ca47bfb61228d968d524fd" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 37807 } } peers { permanent_uuid: "50ea117b4ffa45bc8456eb9ec24594b3" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 41607 } } peers { permanent_uuid: "aab3b242cd3e451e958340a207168137" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 38449 } }
I20260806 04:58:07.209393 20952 ts_tablet_manager.cc:1428] T 0b9b038b47b24c7a80b020b1225b55e9 P 50ea117b4ffa45bc8456eb9ec24594b3: Time spent starting tablet: real 0.012s user 0.006s sys 0.006s
I20260806 04:58:07.210403 20688 heartbeater.cc:503] Master 127.9.96.126:40031 was elected leader, sending a full tablet report...
I20260806 04:58:07.212594 20954 ts_tablet_manager.cc:1428] T 0b9b038b47b24c7a80b020b1225b55e9 P aab3b242cd3e451e958340a207168137: Time spent starting tablet: real 0.017s user 0.007s sys 0.008s
I20260806 04:58:07.213336 20901 heartbeater.cc:503] Master 127.9.96.126:40031 was elected leader, sending a full tablet report...
W20260806 04:58:07.304883 20903 tablet.cc:2369] T 0b9b038b47b24c7a80b020b1225b55e9 P aab3b242cd3e451e958340a207168137: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 04:58:07.311269 20449 tablet.cc:2369] T 0b9b038b47b24c7a80b020b1225b55e9 P 2259df3ac9ca47bfb61228d968d524fd: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 04:58:07.426713 20691 tablet.cc:2369] T 0b9b038b47b24c7a80b020b1225b55e9 P 50ea117b4ffa45bc8456eb9ec24594b3: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 04:58:07.448437 20961 raft_consensus.cc:482] T 0b9b038b47b24c7a80b020b1225b55e9 P 2259df3ac9ca47bfb61228d968d524fd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 04:58:07.448721 20961 raft_consensus.cc:504] T 0b9b038b47b24c7a80b020b1225b55e9 P 2259df3ac9ca47bfb61228d968d524fd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2259df3ac9ca47bfb61228d968d524fd" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 37807 } } peers { permanent_uuid: "50ea117b4ffa45bc8456eb9ec24594b3" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 41607 } } peers { permanent_uuid: "aab3b242cd3e451e958340a207168137" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 38449 } }
I20260806 04:58:07.450325 20961 leader_election.cc:288] T 0b9b038b47b24c7a80b020b1225b55e9 P 2259df3ac9ca47bfb61228d968d524fd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 50ea117b4ffa45bc8456eb9ec24594b3 (127.9.96.66:41607), aab3b242cd3e451e958340a207168137 (127.9.96.67:38449)
I20260806 04:58:07.457643 20947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:58:07.450679 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.96.65:57297 (local address 127.9.96.66:41607)
0806 04:58:07.450857 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0806 04:58:07.450862 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 04:58:07.451494 (+ 632us) server_negotiation.cc:407] Connection header received
0806 04:58:07.451724 (+ 230us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:58:07.451731 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:58:07.451821 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:58:07.451938 (+ 117us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 04:58:07.452633 (+ 695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:07.453556 (+ 923us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:07.455697 (+ 2141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:07.456658 (+ 961us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:07.456798 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:58:07.456874 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0806 04:58:07.457253 (+ 379us) server_negotiation.cc:299] Negotiation successful
0806 04:58:07.457384 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260806 04:58:07.457834 20966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:58:07.450685 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.66:41607 (local address 127.9.96.65:57297)
0806 04:58:07.451330 (+ 645us) negotiation.cc:107] Waiting for socket to connect
0806 04:58:07.451346 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 04:58:07.451579 (+ 233us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:58:07.452051 (+ 472us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:58:07.452054 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:58:07.452078 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 04:58:07.452470 (+ 392us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:58:07.452484 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:58:07.453690 (+ 1206us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:58:07.453695 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:58:07.455515 (+ 1820us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:58:07.455542 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:58:07.456868 (+ 1326us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:58:07.456876 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:58:07.457071 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:58:07.457114 (+ 43us) client_negotiation.cc:769] Sending connection context
0806 04:58:07.457199 (+ 85us) client_negotiation.cc:240] Negotiation successful
0806 04:58:07.457286 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":524,"mutex_wait_us":57,"thread_start_us":352,"threads_started":1}
I20260806 04:58:07.457911 20949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:58:07.451769 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.96.65:51313 (local address 127.9.96.67:38449)
0806 04:58:07.451997 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0806 04:58:07.452004 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 04:58:07.452221 (+ 217us) server_negotiation.cc:407] Connection header received
0806 04:58:07.452371 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:58:07.452376 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:58:07.452460 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:58:07.452567 (+ 107us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 04:58:07.453188 (+ 621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:07.454009 (+ 821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:07.455868 (+ 1859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:07.456830 (+ 962us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:07.456971 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:58:07.457078 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0806 04:58:07.457511 (+ 433us) server_negotiation.cc:299] Negotiation successful
0806 04:58:07.457644 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":95}
I20260806 04:58:07.458256 20967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:58:07.451515 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.67:38449 (local address 127.9.96.65:51313)
0806 04:58:07.452120 (+ 605us) negotiation.cc:107] Waiting for socket to connect
0806 04:58:07.452136 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 04:58:07.452244 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:58:07.452616 (+ 372us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:58:07.452622 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:58:07.452647 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 04:58:07.452994 (+ 347us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:58:07.453005 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:58:07.454152 (+ 1147us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:58:07.454155 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:58:07.455734 (+ 1579us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:58:07.455750 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:58:07.457119 (+ 1369us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:58:07.457126 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:58:07.457333 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:58:07.457375 (+ 42us) client_negotiation.cc:769] Sending connection context
0806 04:58:07.457463 (+ 88us) client_negotiation.cc:240] Negotiation successful
0806 04:58:07.457585 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":476,"thread_start_us":321,"threads_started":1}
I20260806 04:58:07.458940 20609 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b9b038b47b24c7a80b020b1225b55e9" candidate_uuid: "2259df3ac9ca47bfb61228d968d524fd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "50ea117b4ffa45bc8456eb9ec24594b3" is_pre_election: true
I20260806 04:58:07.459292 20840 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b9b038b47b24c7a80b020b1225b55e9" candidate_uuid: "2259df3ac9ca47bfb61228d968d524fd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aab3b242cd3e451e958340a207168137" is_pre_election: true
I20260806 04:58:07.459544 20609 raft_consensus.cc:2454] T 0b9b038b47b24c7a80b020b1225b55e9 P 50ea117b4ffa45bc8456eb9ec24594b3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2259df3ac9ca47bfb61228d968d524fd in term 0.
I20260806 04:58:07.459923 20840 raft_consensus.cc:2454] T 0b9b038b47b24c7a80b020b1225b55e9 P aab3b242cd3e451e958340a207168137 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2259df3ac9ca47bfb61228d968d524fd in term 0.
I20260806 04:58:07.460239 20289 leader_election.cc:302] T 0b9b038b47b24c7a80b020b1225b55e9 P 2259df3ac9ca47bfb61228d968d524fd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2259df3ac9ca47bfb61228d968d524fd, 50ea117b4ffa45bc8456eb9ec24594b3; no voters:
I20260806 04:58:07.460659 20961 raft_consensus.cc:2781] T 0b9b038b47b24c7a80b020b1225b55e9 P 2259df3ac9ca47bfb61228d968d524fd [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 04:58:07.460821 20961 raft_consensus.cc:482] T 0b9b038b47b24c7a80b020b1225b55e9 P 2259df3ac9ca47bfb61228d968d524fd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 04:58:07.460945 20961 raft_consensus.cc:3037] T 0b9b038b47b24c7a80b020b1225b55e9 P 2259df3ac9ca47bfb61228d968d524fd [term 0 FOLLOWER]: Advancing to term 1
I20260806 04:58:07.463218 20961 raft_consensus.cc:504] T 0b9b038b47b24c7a80b020b1225b55e9 P 2259df3ac9ca47bfb61228d968d524fd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2259df3ac9ca47bfb61228d968d524fd" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 37807 } } peers { permanent_uuid: "50ea117b4ffa45bc8456eb9ec24594b3" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 41607 } } peers { permanent_uuid: "aab3b242cd3e451e958340a207168137" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 38449 } }
I20260806 04:58:07.463792 20961 leader_election.cc:288] T 0b9b038b47b24c7a80b020b1225b55e9 P 2259df3ac9ca47bfb61228d968d524fd [CANDIDATE]: Term 1 election: Requested vote from peers 50ea117b4ffa45bc8456eb9ec24594b3 (127.9.96.66:41607), aab3b242cd3e451e958340a207168137 (127.9.96.67:38449)
I20260806 04:58:07.464393 20840 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b9b038b47b24c7a80b020b1225b55e9" candidate_uuid: "2259df3ac9ca47bfb61228d968d524fd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aab3b242cd3e451e958340a207168137"
I20260806 04:58:07.464411 20609 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b9b038b47b24c7a80b020b1225b55e9" candidate_uuid: "2259df3ac9ca47bfb61228d968d524fd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "50ea117b4ffa45bc8456eb9ec24594b3"
I20260806 04:58:07.464573 20840 raft_consensus.cc:3037] T 0b9b038b47b24c7a80b020b1225b55e9 P aab3b242cd3e451e958340a207168137 [term 0 FOLLOWER]: Advancing to term 1
I20260806 04:58:07.464605 20609 raft_consensus.cc:3037] T 0b9b038b47b24c7a80b020b1225b55e9 P 50ea117b4ffa45bc8456eb9ec24594b3 [term 0 FOLLOWER]: Advancing to term 1
I20260806 04:58:07.467073 20840 raft_consensus.cc:2454] T 0b9b038b47b24c7a80b020b1225b55e9 P aab3b242cd3e451e958340a207168137 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2259df3ac9ca47bfb61228d968d524fd in term 1.
I20260806 04:58:07.467163 20609 raft_consensus.cc:2454] T 0b9b038b47b24c7a80b020b1225b55e9 P 50ea117b4ffa45bc8456eb9ec24594b3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2259df3ac9ca47bfb61228d968d524fd in term 1.
I20260806 04:58:07.467569 20289 leader_election.cc:302] T 0b9b038b47b24c7a80b020b1225b55e9 P 2259df3ac9ca47bfb61228d968d524fd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2259df3ac9ca47bfb61228d968d524fd, aab3b242cd3e451e958340a207168137; no voters:
I20260806 04:58:07.467829 20961 raft_consensus.cc:2781] T 0b9b038b47b24c7a80b020b1225b55e9 P 2259df3ac9ca47bfb61228d968d524fd [term 1 FOLLOWER]: Leader election won for term 1
I20260806 04:58:07.468263 20961 raft_consensus.cc:686] T 0b9b038b47b24c7a80b020b1225b55e9 P 2259df3ac9ca47bfb61228d968d524fd [term 1 LEADER]: Becoming Leader. State: Replica: 2259df3ac9ca47bfb61228d968d524fd, State: Running, Role: LEADER
I20260806 04:58:07.468890 20961 consensus_queue.cc:237] T 0b9b038b47b24c7a80b020b1225b55e9 P 2259df3ac9ca47bfb61228d968d524fd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2259df3ac9ca47bfb61228d968d524fd" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 37807 } } peers { permanent_uuid: "50ea117b4ffa45bc8456eb9ec24594b3" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 41607 } } peers { permanent_uuid: "aab3b242cd3e451e958340a207168137" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 38449 } }
I20260806 04:58:07.474686 18960 catalog_manager.cc:5495] T 0b9b038b47b24c7a80b020b1225b55e9 P 2259df3ac9ca47bfb61228d968d524fd reported cstate change: term changed from 0 to 1, leader changed from <none> to 2259df3ac9ca47bfb61228d968d524fd (127.9.96.65). New cstate: current_term: 1 leader_uuid: "2259df3ac9ca47bfb61228d968d524fd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2259df3ac9ca47bfb61228d968d524fd" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 37807 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "50ea117b4ffa45bc8456eb9ec24594b3" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 41607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aab3b242cd3e451e958340a207168137" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 38449 } health_report { overall_health: UNKNOWN } } }
I20260806 04:58:07.525266 20945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:58:07.520107 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56366 (local address 127.9.96.65:37807)
0806 04:58:07.520310 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0806 04:58:07.520314 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 04:58:07.520553 (+ 239us) server_negotiation.cc:407] Connection header received
0806 04:58:07.520697 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:58:07.520702 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:58:07.520775 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:58:07.520895 (+ 120us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 04:58:07.521274 (+ 379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:07.521983 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:07.523202 (+ 1219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:07.523667 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:07.523763 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:58:07.524188 (+ 425us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 04:58:07.524707 (+ 519us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 04:58:07.524817 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0806 04:58:07.524892 (+ 75us) server_negotiation.cc:299] Negotiation successful
0806 04:58:07.524954 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260806 04:58:07.532995 20840 raft_consensus.cc:1261] T 0b9b038b47b24c7a80b020b1225b55e9 P aab3b242cd3e451e958340a207168137 [term 1 FOLLOWER]: Refusing update from remote peer 2259df3ac9ca47bfb61228d968d524fd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 04:58:07.533021 20609 raft_consensus.cc:1261] T 0b9b038b47b24c7a80b020b1225b55e9 P 50ea117b4ffa45bc8456eb9ec24594b3 [term 1 FOLLOWER]: Refusing update from remote peer 2259df3ac9ca47bfb61228d968d524fd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 04:58:07.533911 20968 consensus_queue.cc:1035] T 0b9b038b47b24c7a80b020b1225b55e9 P 2259df3ac9ca47bfb61228d968d524fd [LEADER]: Connected to new peer: Peer: permanent_uuid: "50ea117b4ffa45bc8456eb9ec24594b3" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 41607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 04:58:07.534260 20961 consensus_queue.cc:1035] T 0b9b038b47b24c7a80b020b1225b55e9 P 2259df3ac9ca47bfb61228d968d524fd [LEADER]: Connected to new peer: Peer: permanent_uuid: "aab3b242cd3e451e958340a207168137" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 38449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 04:58:07.568188 18957 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48752:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 04:58:07.568509 18957 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48752:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 04:58:07.575316 18957 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6126c11fed1e4765becce7bd422ce904: Sending DeleteTablet for 3 replicas of tablet 0b9b038b47b24c7a80b020b1225b55e9
I20260806 04:58:07.576841 20574 tablet_service.cc:1514] Processing DeleteTablet for tablet 0b9b038b47b24c7a80b020b1225b55e9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 04:58:07 UTC) from {username='oryx', principal='oryx/127.9.96.126@KRBTEST.COM'} at 127.0.0.1:54552
I20260806 04:58:07.577150 20804 tablet_service.cc:1514] Processing DeleteTablet for tablet 0b9b038b47b24c7a80b020b1225b55e9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 04:58:07 UTC) from {username='oryx', principal='oryx/127.9.96.126@KRBTEST.COM'} at 127.0.0.1:39344
I20260806 04:58:07.577301 20348 tablet_service.cc:1514] Processing DeleteTablet for tablet 0b9b038b47b24c7a80b020b1225b55e9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 04:58:07 UTC) from {username='oryx', principal='oryx/127.9.96.126@KRBTEST.COM'} at 127.0.0.1:56358
I20260806 04:58:07.587900 9601 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu pid 18915
I20260806 04:58:07.588939 19001 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 04:58:07.589987 20988 tablet_replica.cc:330] T 0b9b038b47b24c7a80b020b1225b55e9 P 2259df3ac9ca47bfb61228d968d524fd: stopping tablet replica
I20260806 04:58:07.590555 20988 raft_consensus.cc:2229] T 0b9b038b47b24c7a80b020b1225b55e9 P 2259df3ac9ca47bfb61228d968d524fd [term 1 LEADER]: Raft consensus shutting down.
I20260806 04:58:07.591450 20988 raft_consensus.cc:2258] T 0b9b038b47b24c7a80b020b1225b55e9 P 2259df3ac9ca47bfb61228d968d524fd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 04:58:07.596624 20986 tablet_replica.cc:330] T 0b9b038b47b24c7a80b020b1225b55e9 P 50ea117b4ffa45bc8456eb9ec24594b3: stopping tablet replica
I20260806 04:58:07.597036 20986 raft_consensus.cc:2229] T 0b9b038b47b24c7a80b020b1225b55e9 P 50ea117b4ffa45bc8456eb9ec24594b3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 04:58:07.597522 20986 raft_consensus.cc:2258] T 0b9b038b47b24c7a80b020b1225b55e9 P 50ea117b4ffa45bc8456eb9ec24594b3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 04:58:07.599583 20988 ts_tablet_manager.cc:1905] T 0b9b038b47b24c7a80b020b1225b55e9 P 2259df3ac9ca47bfb61228d968d524fd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 04:58:07.610819 20988 ts_tablet_manager.cc:1918] T 0b9b038b47b24c7a80b020b1225b55e9 P 2259df3ac9ca47bfb61228d968d524fd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 04:58:07.611025 20988 log.cc:1196] T 0b9b038b47b24c7a80b020b1225b55e9 P 2259df3ac9ca47bfb61228d968d524fd: Deleting WAL directory at /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785992227050689-9601-0/minicluster-data/ts-0/wal/wals/0b9b038b47b24c7a80b020b1225b55e9
I20260806 04:58:07.615105 20986 ts_tablet_manager.cc:1905] T 0b9b038b47b24c7a80b020b1225b55e9 P 50ea117b4ffa45bc8456eb9ec24594b3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 04:58:07.615851 20987 tablet_replica.cc:330] T 0b9b038b47b24c7a80b020b1225b55e9 P aab3b242cd3e451e958340a207168137: stopping tablet replica
I20260806 04:58:07.616331 20987 raft_consensus.cc:2229] T 0b9b038b47b24c7a80b020b1225b55e9 P aab3b242cd3e451e958340a207168137 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 04:58:07.616815 20987 raft_consensus.cc:2258] T 0b9b038b47b24c7a80b020b1225b55e9 P aab3b242cd3e451e958340a207168137 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 04:58:07.621600 20987 ts_tablet_manager.cc:1905] T 0b9b038b47b24c7a80b020b1225b55e9 P aab3b242cd3e451e958340a207168137: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 04:58:07.626560 20986 ts_tablet_manager.cc:1918] T 0b9b038b47b24c7a80b020b1225b55e9 P 50ea117b4ffa45bc8456eb9ec24594b3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 04:58:07.627113 20988 ts_tablet_manager.cc:1939] T 0b9b038b47b24c7a80b020b1225b55e9 P 2259df3ac9ca47bfb61228d968d524fd: Deleting consensus metadata
I20260806 04:58:07.628974 20986 log.cc:1196] T 0b9b038b47b24c7a80b020b1225b55e9 P 50ea117b4ffa45bc8456eb9ec24594b3: Deleting WAL directory at /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785992227050689-9601-0/minicluster-data/ts-1/wal/wals/0b9b038b47b24c7a80b020b1225b55e9
I20260806 04:58:07.629633 20986 ts_tablet_manager.cc:1939] T 0b9b038b47b24c7a80b020b1225b55e9 P 50ea117b4ffa45bc8456eb9ec24594b3: Deleting consensus metadata
I20260806 04:58:07.630760 20916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:58:07.582207 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53034 (local address 127.9.96.125:39617)
0806 04:58:07.582364 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0806 04:58:07.582370 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 04:58:07.587055 (+ 4685us) server_negotiation.cc:407] Connection header received
0806 04:58:07.587120 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:58:07.587125 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:58:07.587201 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:58:07.587301 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:58:07.600829 (+ 13528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:07.602063 (+ 1234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:07.612696 (+ 10633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:07.613354 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:07.613444 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:58:07.623562 (+ 10118us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:58:07.623568 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:58:07.623573 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:58:07.623636 (+ 63us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:58:07.627278 (+ 3642us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:58:07.628170 (+ 892us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:58:07.628176 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:58:07.628181 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:58:07.628330 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:58:07.628744 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:58:07.628749 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:58:07.628752 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:58:07.629136 (+ 384us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:58:07.629408 (+ 272us) server_negotiation.cc:1035] Waiting for connection context
0806 04:58:07.630158 (+ 750us) server_negotiation.cc:299] Negotiation successful
0806 04:58:07.630401 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260806 04:58:07.637643 20987 ts_tablet_manager.cc:1918] T 0b9b038b47b24c7a80b020b1225b55e9 P aab3b242cd3e451e958340a207168137: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 04:58:07.637823 20987 log.cc:1196] T 0b9b038b47b24c7a80b020b1225b55e9 P aab3b242cd3e451e958340a207168137: Deleting WAL directory at /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785992227050689-9601-0/minicluster-data/ts-2/wal/wals/0b9b038b47b24c7a80b020b1225b55e9
I20260806 04:58:07.638335 20987 ts_tablet_manager.cc:1939] T 0b9b038b47b24c7a80b020b1225b55e9 P aab3b242cd3e451e958340a207168137: Deleting consensus metadata
I20260806 04:58:07.648553 20905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:58:07.586854 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55198 (local address 127.9.96.124:44477)
0806 04:58:07.587012 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0806 04:58:07.587018 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 04:58:07.631960 (+ 44942us) server_negotiation.cc:407] Connection header received
0806 04:58:07.632237 (+ 277us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:58:07.632245 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:58:07.632340 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:58:07.632445 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:58:07.634339 (+ 1894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:07.635549 (+ 1210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:07.637465 (+ 1916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:07.638153 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:07.638251 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:58:07.641884 (+ 3633us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:58:07.641891 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:58:07.641896 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:58:07.641935 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:58:07.645446 (+ 3511us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:58:07.646268 (+ 822us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:58:07.646274 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:58:07.646278 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:58:07.646420 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:58:07.646823 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:58:07.646828 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:58:07.646831 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:58:07.647201 (+ 370us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:58:07.647556 (+ 355us) server_negotiation.cc:1035] Waiting for connection context
0806 04:58:07.647800 (+ 244us) server_negotiation.cc:299] Negotiation successful
0806 04:58:07.648029 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260806 04:58:07.729686 18939 catalog_manager.cc:4841] TS aab3b242cd3e451e958340a207168137 (127.9.96.67:38449): tablet 0b9b038b47b24c7a80b020b1225b55e9 (table test-table [id=32ceabd633b947b6be332bd29e1530b8]) successfully deleted
I20260806 04:58:07.730394 9601 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu with pid 18915
W20260806 04:58:07.756880 20290 connection.cc:535] server connection from 127.0.0.1:56358 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260806 04:58:07.782814 20905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:58:07.761889 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55204 (local address 127.9.96.124:44477)
0806 04:58:07.762165 (+ 276us) server_negotiation.cc:206] Beginning negotiation
0806 04:58:07.762174 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 04:58:07.763168 (+ 994us) server_negotiation.cc:407] Connection header received
0806 04:58:07.764037 (+ 869us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:58:07.764047 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:58:07.764166 (+ 119us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:58:07.764260 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:58:07.767476 (+ 3216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:07.769038 (+ 1562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:07.771808 (+ 2770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:07.772741 (+ 933us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:07.772862 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:58:07.776286 (+ 3424us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:58:07.776295 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:58:07.776300 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:58:07.776352 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:58:07.779874 (+ 3522us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:58:07.780543 (+ 669us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:58:07.780548 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:58:07.780551 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:58:07.780678 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:58:07.781102 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:58:07.781107 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:58:07.781111 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:58:07.781474 (+ 363us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:58:07.781726 (+ 252us) server_negotiation.cc:1035] Waiting for connection context
0806 04:58:07.782081 (+ 355us) server_negotiation.cc:299] Negotiation successful
0806 04:58:07.782458 (+ 377us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260806 04:58:07.785379 20916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:58:07.762908 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53044 (local address 127.9.96.125:39617)
0806 04:58:07.763094 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0806 04:58:07.763100 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 04:58:07.769065 (+ 5965us) server_negotiation.cc:407] Connection header received
0806 04:58:07.769603 (+ 538us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:58:07.769609 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:58:07.769690 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:58:07.769760 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:58:07.772402 (+ 2642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:07.773535 (+ 1133us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:07.775924 (+ 2389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:07.776484 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:07.776632 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:58:07.779270 (+ 2638us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:58:07.779275 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:58:07.779278 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:58:07.779309 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:58:07.782562 (+ 3253us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:58:07.783254 (+ 692us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:58:07.783259 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:58:07.783262 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:58:07.783442 (+ 180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:58:07.783815 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:58:07.783821 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:58:07.783843 (+ 22us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:58:07.784214 (+ 371us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:58:07.784517 (+ 303us) server_negotiation.cc:1035] Waiting for connection context
0806 04:58:07.784852 (+ 335us) server_negotiation.cc:299] Negotiation successful
0806 04:58:07.785092 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260806 04:58:08.561779 21006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:58:08.561075 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:40031 (local address 127.9.96.66:34227)
0806 04:58:08.561469 (+ 394us) negotiation.cc:107] Waiting for socket to connect
0806 04:58:08.561573 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.96.126:40031: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":309,"thread_start_us":217,"threads_started":1}
W20260806 04:58:08.562355 20688 heartbeater.cc:644] Failed to heartbeat to 127.9.96.126:40031 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.9.96.126:40031: connect: Connection refused (error 111)
I20260806 04:58:08.563752 21008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:58:08.562756 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:40031 (local address 127.9.96.67:50785)
0806 04:58:08.563158 (+ 402us) negotiation.cc:107] Waiting for socket to connect
0806 04:58:08.563287 (+ 129us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.96.126:40031: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":302,"thread_start_us":599,"threads_started":1}
I20260806 04:58:08.564013 21009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:58:08.563194 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:40031 (local address 127.9.96.65:58455)
0806 04:58:08.563656 (+ 462us) negotiation.cc:107] Waiting for socket to connect
0806 04:58:08.563809 (+ 153us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.96.126:40031: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":342,"thread_start_us":283,"threads_started":1}
W20260806 04:58:08.564247 20901 heartbeater.cc:644] Failed to heartbeat to 127.9.96.126:40031 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.9.96.126:40031: connect: Connection refused (error 111)
W20260806 04:58:08.564321 20445 heartbeater.cc:644] Failed to heartbeat to 127.9.96.126:40031 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.9.96.126:40031: connect: Connection refused (error 111)
I20260806 04:58:09.075604 21010 raft_consensus.cc:482] T 00000000000000000000000000000000 P 89e9df10c70d4ddaacd2a30fff7a3b2d [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6126c11fed1e4765becce7bd422ce904)
I20260806 04:58:09.075886 21010 raft_consensus.cc:504] T 00000000000000000000000000000000 P 89e9df10c70d4ddaacd2a30fff7a3b2d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "6126c11fed1e4765becce7bd422ce904" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 40031 } } peers { permanent_uuid: "89e9df10c70d4ddaacd2a30fff7a3b2d" member_type: VOTER last_known_addr { host: "127.9.96.125" port: 39617 } attrs { promote: false } } peers { permanent_uuid: "50f00ca72a174ac4a6a7f39da3065462" member_type: VOTER last_known_addr { host: "127.9.96.124" port: 44477 } attrs { promote: false } }
I20260806 04:58:09.077214 21010 leader_election.cc:288] T 00000000000000000000000000000000 P 89e9df10c70d4ddaacd2a30fff7a3b2d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6126c11fed1e4765becce7bd422ce904 (127.9.96.126:40031), 50f00ca72a174ac4a6a7f39da3065462 (127.9.96.124:44477)
I20260806 04:58:09.078173 21011 raft_consensus.cc:482] T 00000000000000000000000000000000 P 50f00ca72a174ac4a6a7f39da3065462 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6126c11fed1e4765becce7bd422ce904)
I20260806 04:58:09.078598 21011 raft_consensus.cc:504] T 00000000000000000000000000000000 P 50f00ca72a174ac4a6a7f39da3065462 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "6126c11fed1e4765becce7bd422ce904" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 40031 } } peers { permanent_uuid: "89e9df10c70d4ddaacd2a30fff7a3b2d" member_type: VOTER last_known_addr { host: "127.9.96.125" port: 39617 } attrs { promote: false } } peers { permanent_uuid: "50f00ca72a174ac4a6a7f39da3065462" member_type: VOTER last_known_addr { host: "127.9.96.124" port: 44477 } attrs { promote: false } }
I20260806 04:58:09.080457 21011 leader_election.cc:288] T 00000000000000000000000000000000 P 50f00ca72a174ac4a6a7f39da3065462 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6126c11fed1e4765becce7bd422ce904 (127.9.96.126:40031), 89e9df10c70d4ddaacd2a30fff7a3b2d (127.9.96.125:39617)
I20260806 04:58:09.082065 21016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:58:09.080616 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:40031 (local address 127.0.0.1:49180)
0806 04:58:09.081499 (+ 883us) negotiation.cc:107] Waiting for socket to connect
0806 04:58:09.081711 (+ 212us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.96.126:40031: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":767,"thread_start_us":647,"threads_started":1}
I20260806 04:58:09.083801 21013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:58:09.077421 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:40031 (local address 127.0.0.1:49172)
0806 04:58:09.083168 (+ 5747us) negotiation.cc:107] Waiting for socket to connect
0806 04:58:09.083505 (+ 337us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.96.126:40031: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":5546,"thread_start_us":5780,"threads_started":1}
W20260806 04:58:09.083783 20149 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.9.96.126:40031: connect: Connection refused (error 111)
W20260806 04:58:09.084057 19843 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.9.96.126:40031: connect: Connection refused (error 111)
I20260806 04:58:09.092120 21013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:58:09.091718 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:40031 (local address 127.0.0.1:49182)
0806 04:58:09.091872 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0806 04:58:09.091978 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.96.126:40031: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260806 04:58:09.092419 19843 leader_election.cc:334] T 00000000000000000000000000000000 P 89e9df10c70d4ddaacd2a30fff7a3b2d [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 6126c11fed1e4765becce7bd422ce904 (127.9.96.126:40031): Network error: Client connection negotiation failed: client connection to 127.9.96.126:40031: connect: Connection refused (error 111)
I20260806 04:58:09.093830 21012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:58:09.077491 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.124:44477 (local address 127.0.0.1:55218)
0806 04:58:09.083487 (+ 5996us) negotiation.cc:107] Waiting for socket to connect
0806 04:58:09.083510 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0806 04:58:09.083702 (+ 192us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:58:09.084338 (+ 636us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:58:09.084343 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:58:09.084372 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 04:58:09.084853 (+ 481us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:58:09.084882 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:58:09.087213 (+ 2331us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:58:09.087222 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:58:09.090185 (+ 2963us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:58:09.090206 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:58:09.092266 (+ 2060us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:58:09.092272 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:58:09.092510 (+ 238us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:58:09.092551 (+ 41us) client_negotiation.cc:769] Sending connection context
0806 04:58:09.093311 (+ 760us) client_negotiation.cc:240] Negotiation successful
0806 04:58:09.093429 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5856,"mutex_wait_us":1,"thread_start_us":5322,"threads_started":1}
I20260806 04:58:09.093868 21017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:58:09.093260 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:40031 (local address 127.0.0.1:49198)
0806 04:58:09.093416 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0806 04:58:09.093589 (+ 173us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.96.126:40031: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260806 04:58:09.094177 21016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:58:09.081618 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.125:39617 (local address 127.0.0.1:53052)
0806 04:58:09.082417 (+ 799us) negotiation.cc:107] Waiting for socket to connect
0806 04:58:09.082433 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 04:58:09.082570 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:58:09.084800 (+ 2230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:58:09.084804 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:58:09.084833 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 04:58:09.085285 (+ 452us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:58:09.085405 (+ 120us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:58:09.087970 (+ 2565us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:58:09.087977 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:58:09.090942 (+ 2965us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:58:09.090966 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:58:09.093239 (+ 2273us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:58:09.093244 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:58:09.093419 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:58:09.093457 (+ 38us) client_negotiation.cc:769] Sending connection context
0806 04:58:09.093561 (+ 104us) client_negotiation.cc:240] Negotiation successful
0806 04:58:09.093714 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":681,"mutex_wait_us":4,"thread_start_us":1873,"threads_started":1}
I20260806 04:58:09.094192 21018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:58:09.082636 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53052 (local address 127.9.96.125:39617)
0806 04:58:09.084244 (+ 1608us) server_negotiation.cc:206] Beginning negotiation
0806 04:58:09.084250 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 04:58:09.084368 (+ 118us) server_negotiation.cc:407] Connection header received
0806 04:58:09.084487 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:58:09.084495 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:58:09.084595 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:58:09.085627 (+ 1032us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 04:58:09.086052 (+ 425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:09.087769 (+ 1717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:09.091314 (+ 3545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:09.093071 (+ 1757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:09.093162 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:58:09.093230 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0806 04:58:09.093693 (+ 463us) server_negotiation.cc:299] Negotiation successful
0806 04:58:09.093837 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":238,"server-negotiator.queue_time_us":1451,"thread_start_us":1549,"threads_started":1}
W20260806 04:58:09.094326 20149 leader_election.cc:334] T 00000000000000000000000000000000 P 50f00ca72a174ac4a6a7f39da3065462 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 6126c11fed1e4765becce7bd422ce904 (127.9.96.126:40031): Network error: Client connection negotiation failed: client connection to 127.9.96.126:40031: connect: Connection refused (error 111)
I20260806 04:58:09.094473 21014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:58:09.078095 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55218 (local address 127.9.96.124:44477)
0806 04:58:09.081099 (+ 3004us) server_negotiation.cc:206] Beginning negotiation
0806 04:58:09.081109 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 04:58:09.083589 (+ 2480us) server_negotiation.cc:407] Connection header received
0806 04:58:09.083901 (+ 312us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:58:09.083907 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:58:09.083980 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:58:09.084061 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 04:58:09.085169 (+ 1108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:09.086998 (+ 1829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:09.090424 (+ 3426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:09.091922 (+ 1498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:09.092016 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:58:09.092105 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0806 04:58:09.093346 (+ 1241us) server_negotiation.cc:299] Negotiation successful
0806 04:58:09.093488 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":29,"server-negotiator.queue_time_us":2814,"thread_start_us":2699,"threads_started":1}
I20260806 04:58:09.095070 19885 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "50f00ca72a174ac4a6a7f39da3065462" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "89e9df10c70d4ddaacd2a30fff7a3b2d" is_pre_election: true
I20260806 04:58:09.095083 20194 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "89e9df10c70d4ddaacd2a30fff7a3b2d" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "50f00ca72a174ac4a6a7f39da3065462" is_pre_election: true
I20260806 04:58:09.095367 19885 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 89e9df10c70d4ddaacd2a30fff7a3b2d [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50f00ca72a174ac4a6a7f39da3065462 in term 1.
I20260806 04:58:09.095465 20194 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 50f00ca72a174ac4a6a7f39da3065462 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 89e9df10c70d4ddaacd2a30fff7a3b2d in term 1.
I20260806 04:58:09.095963 20149 leader_election.cc:302] T 00000000000000000000000000000000 P 50f00ca72a174ac4a6a7f39da3065462 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 50f00ca72a174ac4a6a7f39da3065462, 89e9df10c70d4ddaacd2a30fff7a3b2d; no voters: 6126c11fed1e4765becce7bd422ce904
I20260806 04:58:09.095973 19846 leader_election.cc:302] T 00000000000000000000000000000000 P 89e9df10c70d4ddaacd2a30fff7a3b2d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 50f00ca72a174ac4a6a7f39da3065462, 89e9df10c70d4ddaacd2a30fff7a3b2d; no voters: 6126c11fed1e4765becce7bd422ce904
I20260806 04:58:09.096334 21011 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 50f00ca72a174ac4a6a7f39da3065462 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260806 04:58:09.096377 21010 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 89e9df10c70d4ddaacd2a30fff7a3b2d [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260806 04:58:09.096469 21011 raft_consensus.cc:482] T 00000000000000000000000000000000 P 50f00ca72a174ac4a6a7f39da3065462 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 6126c11fed1e4765becce7bd422ce904)
I20260806 04:58:09.096510 21010 raft_consensus.cc:482] T 00000000000000000000000000000000 P 89e9df10c70d4ddaacd2a30fff7a3b2d [term 1 FOLLOWER]: Starting leader election (detected failure of leader 6126c11fed1e4765becce7bd422ce904)
I20260806 04:58:09.096565 21011 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 50f00ca72a174ac4a6a7f39da3065462 [term 1 FOLLOWER]: Advancing to term 2
I20260806 04:58:09.096606 21010 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 89e9df10c70d4ddaacd2a30fff7a3b2d [term 1 FOLLOWER]: Advancing to term 2
I20260806 04:58:09.098588 21010 raft_consensus.cc:504] T 00000000000000000000000000000000 P 89e9df10c70d4ddaacd2a30fff7a3b2d [term 2 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "6126c11fed1e4765becce7bd422ce904" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 40031 } } peers { permanent_uuid: "89e9df10c70d4ddaacd2a30fff7a3b2d" member_type: VOTER last_known_addr { host: "127.9.96.125" port: 39617 } attrs { promote: false } } peers { permanent_uuid: "50f00ca72a174ac4a6a7f39da3065462" member_type: VOTER last_known_addr { host: "127.9.96.124" port: 44477 } attrs { promote: false } }
I20260806 04:58:09.098585 21011 raft_consensus.cc:504] T 00000000000000000000000000000000 P 50f00ca72a174ac4a6a7f39da3065462 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "6126c11fed1e4765becce7bd422ce904" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 40031 } } peers { permanent_uuid: "89e9df10c70d4ddaacd2a30fff7a3b2d" member_type: VOTER last_known_addr { host: "127.9.96.125" port: 39617 } attrs { promote: false } } peers { permanent_uuid: "50f00ca72a174ac4a6a7f39da3065462" member_type: VOTER last_known_addr { host: "127.9.96.124" port: 44477 } attrs { promote: false } }
I20260806 04:58:09.099150 21011 leader_election.cc:288] T 00000000000000000000000000000000 P 50f00ca72a174ac4a6a7f39da3065462 [CANDIDATE]: Term 2 election: Requested vote from peers 6126c11fed1e4765becce7bd422ce904 (127.9.96.126:40031), 89e9df10c70d4ddaacd2a30fff7a3b2d (127.9.96.125:39617)
I20260806 04:58:09.099206 21010 leader_election.cc:288] T 00000000000000000000000000000000 P 89e9df10c70d4ddaacd2a30fff7a3b2d [CANDIDATE]: Term 2 election: Requested vote from peers 6126c11fed1e4765becce7bd422ce904 (127.9.96.126:40031), 50f00ca72a174ac4a6a7f39da3065462 (127.9.96.124:44477)
I20260806 04:58:09.099684 21012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:58:09.099301 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:40031 (local address 127.0.0.1:49202)
0806 04:58:09.099463 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0806 04:58:09.099556 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.96.126:40031: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260806 04:58:09.099623 20194 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "89e9df10c70d4ddaacd2a30fff7a3b2d" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "50f00ca72a174ac4a6a7f39da3065462"
I20260806 04:58:09.100063 20194 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 50f00ca72a174ac4a6a7f39da3065462 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 89e9df10c70d4ddaacd2a30fff7a3b2d in current term 2: Already voted for candidate 50f00ca72a174ac4a6a7f39da3065462 in this term.
I20260806 04:58:09.100284 19885 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "50f00ca72a174ac4a6a7f39da3065462" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "89e9df10c70d4ddaacd2a30fff7a3b2d"
I20260806 04:58:09.100412 21016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:58:09.099627 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:40031 (local address 127.0.0.1:49218)
0806 04:58:09.099796 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0806 04:58:09.099897 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.96.126:40031: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
I20260806 04:58:09.100570 19885 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 89e9df10c70d4ddaacd2a30fff7a3b2d [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 50f00ca72a174ac4a6a7f39da3065462 in current term 2: Already voted for candidate 89e9df10c70d4ddaacd2a30fff7a3b2d in this term.
I20260806 04:58:09.101044 21012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:58:09.100592 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:40031 (local address 127.0.0.1:49226)
0806 04:58:09.100840 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0806 04:58:09.100921 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.96.126:40031: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":159}
I20260806 04:58:09.101569 21016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:58:09.101276 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:40031 (local address 127.0.0.1:49232)
0806 04:58:09.101408 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0806 04:58:09.101479 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.96.126:40031: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260806 04:58:09.101758 20149 leader_election.cc:334] T 00000000000000000000000000000000 P 50f00ca72a174ac4a6a7f39da3065462 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 6126c11fed1e4765becce7bd422ce904 (127.9.96.126:40031): Network error: Client connection negotiation failed: client connection to 127.9.96.126:40031: connect: Connection refused (error 111)
I20260806 04:58:09.101917 20149 leader_election.cc:302] T 00000000000000000000000000000000 P 50f00ca72a174ac4a6a7f39da3065462 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50f00ca72a174ac4a6a7f39da3065462; no voters: 6126c11fed1e4765becce7bd422ce904, 89e9df10c70d4ddaacd2a30fff7a3b2d
I20260806 04:58:09.102166 21011 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 50f00ca72a174ac4a6a7f39da3065462 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
W20260806 04:58:09.102993 19843 leader_election.cc:334] T 00000000000000000000000000000000 P 89e9df10c70d4ddaacd2a30fff7a3b2d [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 6126c11fed1e4765becce7bd422ce904 (127.9.96.126:40031): Network error: Client connection negotiation failed: client connection to 127.9.96.126:40031: connect: Connection refused (error 111)
I20260806 04:58:09.103641 19843 leader_election.cc:302] T 00000000000000000000000000000000 P 89e9df10c70d4ddaacd2a30fff7a3b2d [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 89e9df10c70d4ddaacd2a30fff7a3b2d; no voters: 50f00ca72a174ac4a6a7f39da3065462, 6126c11fed1e4765becce7bd422ce904
I20260806 04:58:09.103976 21010 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 89e9df10c70d4ddaacd2a30fff7a3b2d [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260806 04:58:09.563779 21027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:58:09.562989 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:40031 (local address 127.9.96.66:46619)
0806 04:58:09.563477 (+ 488us) negotiation.cc:107] Waiting for socket to connect
0806 04:58:09.563580 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.96.126:40031: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":332,"thread_start_us":242,"threads_started":1}
I20260806 04:58:09.565680 21028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:58:09.564899 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:40031 (local address 127.9.96.67:45203)
0806 04:58:09.565349 (+ 450us) negotiation.cc:107] Waiting for socket to connect
0806 04:58:09.565453 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.96.126:40031: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":338,"thread_start_us":555,"threads_started":1}
I20260806 04:58:09.566646 21029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:58:09.564910 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:40031 (local address 127.9.96.65:56311)
0806 04:58:09.566316 (+ 1406us) negotiation.cc:107] Waiting for socket to connect
0806 04:58:09.566428 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.96.126:40031: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1288,"thread_start_us":1196,"threads_started":1}
I20260806 04:58:10.566030 21030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:58:10.564739 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:40031 (local address 127.9.96.66:47819)
0806 04:58:10.565739 (+ 1000us) negotiation.cc:107] Waiting for socket to connect
0806 04:58:10.565841 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.96.126:40031: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":877,"threads_started":1}
I20260806 04:58:10.568218 21031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:58:10.567294 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:40031 (local address 127.9.96.67:34589)
0806 04:58:10.567877 (+ 583us) negotiation.cc:107] Waiting for socket to connect
0806 04:58:10.567968 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.96.126:40031: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":459,"thread_start_us":360,"threads_started":1}
I20260806 04:58:10.568732 21032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:58:10.567944 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:40031 (local address 127.9.96.65:50867)
0806 04:58:10.568527 (+ 583us) negotiation.cc:107] Waiting for socket to connect
0806 04:58:10.568601 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.96.126:40031: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":467,"threads_started":1}
I20260806 04:58:10.709642 21033 raft_consensus.cc:482] T 00000000000000000000000000000000 P 89e9df10c70d4ddaacd2a30fff7a3b2d [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 04:58:10.709887 21033 raft_consensus.cc:504] T 00000000000000000000000000000000 P 89e9df10c70d4ddaacd2a30fff7a3b2d [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "6126c11fed1e4765becce7bd422ce904" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 40031 } } peers { permanent_uuid: "89e9df10c70d4ddaacd2a30fff7a3b2d" member_type: VOTER last_known_addr { host: "127.9.96.125" port: 39617 } attrs { promote: false } } peers { permanent_uuid: "50f00ca72a174ac4a6a7f39da3065462" member_type: VOTER last_known_addr { host: "127.9.96.124" port: 44477 } attrs { promote: false } }
I20260806 04:58:10.710604 21033 leader_election.cc:288] T 00000000000000000000000000000000 P 89e9df10c70d4ddaacd2a30fff7a3b2d [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6126c11fed1e4765becce7bd422ce904 (127.9.96.126:40031), 50f00ca72a174ac4a6a7f39da3065462 (127.9.96.124:44477)
I20260806 04:58:10.711166 20194 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "89e9df10c70d4ddaacd2a30fff7a3b2d" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "50f00ca72a174ac4a6a7f39da3065462" is_pre_election: true
I20260806 04:58:10.711467 20194 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 50f00ca72a174ac4a6a7f39da3065462 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 89e9df10c70d4ddaacd2a30fff7a3b2d in term 2.
I20260806 04:58:10.711705 21034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:58:10.710804 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:40031 (local address 127.0.0.1:59796)
0806 04:58:10.711324 (+ 520us) negotiation.cc:107] Waiting for socket to connect
0806 04:58:10.711483 (+ 159us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.96.126:40031: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":393,"thread_start_us":273,"threads_started":1}
I20260806 04:58:10.712025 19846 leader_election.cc:302] T 00000000000000000000000000000000 P 89e9df10c70d4ddaacd2a30fff7a3b2d [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50f00ca72a174ac4a6a7f39da3065462, 89e9df10c70d4ddaacd2a30fff7a3b2d; no voters:
I20260806 04:58:10.712337 21033 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 89e9df10c70d4ddaacd2a30fff7a3b2d [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260806 04:58:10.712476 21033 raft_consensus.cc:482] T 00000000000000000000000000000000 P 89e9df10c70d4ddaacd2a30fff7a3b2d [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 04:58:10.712602 21033 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 89e9df10c70d4ddaacd2a30fff7a3b2d [term 2 FOLLOWER]: Advancing to term 3
I20260806 04:58:10.713160 21034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:58:10.712806 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:40031 (local address 127.0.0.1:59810)
0806 04:58:10.712957 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0806 04:58:10.713038 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.96.126:40031: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260806 04:58:10.713351 19843 leader_election.cc:334] T 00000000000000000000000000000000 P 89e9df10c70d4ddaacd2a30fff7a3b2d [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 6126c11fed1e4765becce7bd422ce904 (127.9.96.126:40031): Network error: Client connection negotiation failed: client connection to 127.9.96.126:40031: connect: Connection refused (error 111)
I20260806 04:58:10.714885 21033 raft_consensus.cc:504] T 00000000000000000000000000000000 P 89e9df10c70d4ddaacd2a30fff7a3b2d [term 3 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "6126c11fed1e4765becce7bd422ce904" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 40031 } } peers { permanent_uuid: "89e9df10c70d4ddaacd2a30fff7a3b2d" member_type: VOTER last_known_addr { host: "127.9.96.125" port: 39617 } attrs { promote: false } } peers { permanent_uuid: "50f00ca72a174ac4a6a7f39da3065462" member_type: VOTER last_known_addr { host: "127.9.96.124" port: 44477 } attrs { promote: false } }
I20260806 04:58:10.715477 21033 leader_election.cc:288] T 00000000000000000000000000000000 P 89e9df10c70d4ddaacd2a30fff7a3b2d [CANDIDATE]: Term 3 election: Requested vote from peers 6126c11fed1e4765becce7bd422ce904 (127.9.96.126:40031), 50f00ca72a174ac4a6a7f39da3065462 (127.9.96.124:44477)
I20260806 04:58:10.715926 21034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:58:10.715545 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:40031 (local address 127.0.0.1:59820)
0806 04:58:10.715749 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0806 04:58:10.715816 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.96.126:40031: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":88}
I20260806 04:58:10.715927 20194 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "89e9df10c70d4ddaacd2a30fff7a3b2d" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "50f00ca72a174ac4a6a7f39da3065462"
I20260806 04:58:10.716130 20194 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 50f00ca72a174ac4a6a7f39da3065462 [term 2 FOLLOWER]: Advancing to term 3
I20260806 04:58:10.716833 21034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:58:10.716563 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:40031 (local address 127.0.0.1:59830)
0806 04:58:10.716679 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0806 04:58:10.716735 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.96.126:40031: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260806 04:58:10.717031 19843 leader_election.cc:334] T 00000000000000000000000000000000 P 89e9df10c70d4ddaacd2a30fff7a3b2d [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 6126c11fed1e4765becce7bd422ce904 (127.9.96.126:40031): Network error: Client connection negotiation failed: client connection to 127.9.96.126:40031: connect: Connection refused (error 111)
I20260806 04:58:10.718433 20194 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 50f00ca72a174ac4a6a7f39da3065462 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 89e9df10c70d4ddaacd2a30fff7a3b2d in term 3.
I20260806 04:58:10.718844 19846 leader_election.cc:302] T 00000000000000000000000000000000 P 89e9df10c70d4ddaacd2a30fff7a3b2d [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 50f00ca72a174ac4a6a7f39da3065462, 89e9df10c70d4ddaacd2a30fff7a3b2d; no voters: 6126c11fed1e4765becce7bd422ce904
I20260806 04:58:10.719112 21033 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 89e9df10c70d4ddaacd2a30fff7a3b2d [term 3 FOLLOWER]: Leader election won for term 3
I20260806 04:58:10.719554 21033 raft_consensus.cc:686] T 00000000000000000000000000000000 P 89e9df10c70d4ddaacd2a30fff7a3b2d [term 3 LEADER]: Becoming Leader. State: Replica: 89e9df10c70d4ddaacd2a30fff7a3b2d, State: Running, Role: LEADER
I20260806 04:58:10.719936 21033 consensus_queue.cc:237] T 00000000000000000000000000000000 P 89e9df10c70d4ddaacd2a30fff7a3b2d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "6126c11fed1e4765becce7bd422ce904" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 40031 } } peers { permanent_uuid: "89e9df10c70d4ddaacd2a30fff7a3b2d" member_type: VOTER last_known_addr { host: "127.9.96.125" port: 39617 } attrs { promote: false } } peers { permanent_uuid: "50f00ca72a174ac4a6a7f39da3065462" member_type: VOTER last_known_addr { host: "127.9.96.124" port: 44477 } attrs { promote: false } }
I20260806 04:58:10.721786 21037 sys_catalog.cc:455] T 00000000000000000000000000000000 P 89e9df10c70d4ddaacd2a30fff7a3b2d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 89e9df10c70d4ddaacd2a30fff7a3b2d. Latest consensus state: current_term: 3 leader_uuid: "89e9df10c70d4ddaacd2a30fff7a3b2d" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "6126c11fed1e4765becce7bd422ce904" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 40031 } } peers { permanent_uuid: "89e9df10c70d4ddaacd2a30fff7a3b2d" member_type: VOTER last_known_addr { host: "127.9.96.125" port: 39617 } attrs { promote: false } } peers { permanent_uuid: "50f00ca72a174ac4a6a7f39da3065462" member_type: VOTER last_known_addr { host: "127.9.96.124" port: 44477 } attrs { promote: false } } }
I20260806 04:58:10.722291 21037 sys_catalog.cc:458] T 00000000000000000000000000000000 P 89e9df10c70d4ddaacd2a30fff7a3b2d [sys.catalog]: This master's current role is: LEADER
I20260806 04:58:10.722939 21039 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 04:58:10.726791 21039 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 04:58:10.727667 21039 catalog_manager.cc:1259] Loaded cluster ID: b1cd49f4c4814e84b0f51af05de8eeee
I20260806 04:58:10.727737 21039 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 04:58:10.728909 21039 catalog_manager.cc:1504] Loading token signing keys...
I20260806 04:58:10.729723 21039 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 89e9df10c70d4ddaacd2a30fff7a3b2d: Loaded TSK: 0
I20260806 04:58:10.730562 21039 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 04:58:10.810348 19865 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53044:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 04:58:10.817538 19865 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 04:58:10.821640 21034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:58:10.821216 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:40031 (local address 127.0.0.1:59858)
0806 04:58:10.821405 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0806 04:58:10.821478 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.96.126:40031: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":93}
I20260806 04:58:10.821792 20194 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 50f00ca72a174ac4a6a7f39da3065462 [term 3 FOLLOWER]: Refusing update from remote peer 89e9df10c70d4ddaacd2a30fff7a3b2d: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 3 index: 14. (index mismatch)
I20260806 04:58:10.822548 21037 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 89e9df10c70d4ddaacd2a30fff7a3b2d [LEADER]: Connected to new peer: Peer: permanent_uuid: "50f00ca72a174ac4a6a7f39da3065462" member_type: VOTER last_known_addr { host: "127.9.96.124" port: 44477 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260806 04:58:10.822774 21034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:58:10.822321 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:40031 (local address 127.0.0.1:59862)
0806 04:58:10.822524 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0806 04:58:10.822641 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.96.126:40031: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260806 04:58:10.823064 19843 consensus_peers.cc:487] T 00000000000000000000000000000000 P 89e9df10c70d4ddaacd2a30fff7a3b2d -> Peer 6126c11fed1e4765becce7bd422ce904 (127.9.96.126:40031): Couldn't send request to peer 6126c11fed1e4765becce7bd422ce904. Status: Network error: Client connection negotiation failed: client connection to 127.9.96.126:40031: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260806 04:58:10.825119 21042 sys_catalog.cc:455] T 00000000000000000000000000000000 P 50f00ca72a174ac4a6a7f39da3065462 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 89e9df10c70d4ddaacd2a30fff7a3b2d. Latest consensus state: current_term: 3 leader_uuid: "89e9df10c70d4ddaacd2a30fff7a3b2d" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "6126c11fed1e4765becce7bd422ce904" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 40031 } } peers { permanent_uuid: "89e9df10c70d4ddaacd2a30fff7a3b2d" member_type: VOTER last_known_addr { host: "127.9.96.125" port: 39617 } attrs { promote: false } } peers { permanent_uuid: "50f00ca72a174ac4a6a7f39da3065462" member_type: VOTER last_known_addr { host: "127.9.96.124" port: 44477 } attrs { promote: false } } }
I20260806 04:58:10.825323 21042 sys_catalog.cc:458] T 00000000000000000000000000000000 P 50f00ca72a174ac4a6a7f39da3065462 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 04:58:10.827350 21033 sys_catalog.cc:455] T 00000000000000000000000000000000 P 89e9df10c70d4ddaacd2a30fff7a3b2d [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "89e9df10c70d4ddaacd2a30fff7a3b2d" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "6126c11fed1e4765becce7bd422ce904" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 40031 } } peers { permanent_uuid: "89e9df10c70d4ddaacd2a30fff7a3b2d" member_type: VOTER last_known_addr { host: "127.9.96.125" port: 39617 } attrs { promote: false } } peers { permanent_uuid: "50f00ca72a174ac4a6a7f39da3065462" member_type: VOTER last_known_addr { host: "127.9.96.124" port: 44477 } attrs { promote: false } } }
I20260806 04:58:10.827726 21033 sys_catalog.cc:458] T 00000000000000000000000000000000 P 89e9df10c70d4ddaacd2a30fff7a3b2d [sys.catalog]: This master's current role is: LEADER
I20260806 04:58:10.828058 21043 mvcc.cc:205] Tried to move back new op lower bound from 7315424423199330304 to 7315424422793019392. Current Snapshot: MvccSnapshot[applied={T|T < 7315424423199330304}]
I20260806 04:58:10.829373 21042 sys_catalog.cc:455] T 00000000000000000000000000000000 P 50f00ca72a174ac4a6a7f39da3065462 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "89e9df10c70d4ddaacd2a30fff7a3b2d" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "6126c11fed1e4765becce7bd422ce904" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 40031 } } peers { permanent_uuid: "89e9df10c70d4ddaacd2a30fff7a3b2d" member_type: VOTER last_known_addr { host: "127.9.96.125" port: 39617 } attrs { promote: false } } peers { permanent_uuid: "50f00ca72a174ac4a6a7f39da3065462" member_type: VOTER last_known_addr { host: "127.9.96.124" port: 44477 } attrs { promote: false } } }
I20260806 04:58:10.829625 21042 sys_catalog.cc:458] T 00000000000000000000000000000000 P 50f00ca72a174ac4a6a7f39da3065462 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 04:58:10.851578 21048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:58:10.836190 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58126 (local address 127.9.96.67:38449)
0806 04:58:10.845449 (+ 9259us) server_negotiation.cc:206] Beginning negotiation
0806 04:58:10.845456 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 04:58:10.845476 (+ 20us) server_negotiation.cc:407] Connection header received
0806 04:58:10.845550 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:58:10.845555 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:58:10.845627 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:58:10.845748 (+ 121us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 04:58:10.846261 (+ 513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:10.847162 (+ 901us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:10.849661 (+ 2499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:10.850590 (+ 929us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:10.850743 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:58:10.850856 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0806 04:58:10.851119 (+ 263us) server_negotiation.cc:299] Negotiation successful
0806 04:58:10.851257 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9116,"thread_start_us":8990,"threads_started":1}
I20260806 04:58:10.851598 21034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:58:10.836014 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.67:38449 (local address 127.0.0.1:58126)
0806 04:58:10.836176 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0806 04:58:10.836197 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0806 04:58:10.836335 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:58:10.845765 (+ 9430us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:58:10.845769 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:58:10.845795 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 04:58:10.846083 (+ 288us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:58:10.846091 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:58:10.847304 (+ 1213us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:58:10.847308 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:58:10.849498 (+ 2190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:58:10.849514 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:58:10.850756 (+ 1242us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:58:10.850762 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:58:10.850955 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:58:10.850980 (+ 25us) client_negotiation.cc:769] Sending connection context
0806 04:58:10.851083 (+ 103us) client_negotiation.cc:240] Negotiation successful
0806 04:58:10.851217 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":51,"mutex_wait_us":216}
I20260806 04:58:10.851933 21049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:58:10.836429 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.66:41607 (local address 127.0.0.1:44900)
0806 04:58:10.837017 (+ 588us) negotiation.cc:107] Waiting for socket to connect
0806 04:58:10.837034 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 04:58:10.837148 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:58:10.844878 (+ 7730us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:58:10.844883 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:58:10.844915 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 04:58:10.845402 (+ 487us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:58:10.845414 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:58:10.846890 (+ 1476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:58:10.846897 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:58:10.849340 (+ 2443us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:58:10.849356 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:58:10.850938 (+ 1582us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:58:10.850945 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:58:10.851129 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:58:10.851166 (+ 37us) client_negotiation.cc:769] Sending connection context
0806 04:58:10.851239 (+ 73us) client_negotiation.cc:240] Negotiation successful
0806 04:58:10.851413 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":445,"mutex_wait_us":49,"thread_start_us":313,"threads_started":1}
I20260806 04:58:10.852434 21054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:58:10.838526 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.65:37807 (local address 127.0.0.1:44068)
0806 04:58:10.839004 (+ 478us) negotiation.cc:107] Waiting for socket to connect
0806 04:58:10.839027 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0806 04:58:10.839140 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:58:10.844688 (+ 5548us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:58:10.844696 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:58:10.844732 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 04:58:10.845281 (+ 549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:58:10.845295 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:58:10.847397 (+ 2102us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:58:10.847408 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:58:10.849794 (+ 2386us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:58:10.849808 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:58:10.851609 (+ 1801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:58:10.851619 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:58:10.851805 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:58:10.851840 (+ 35us) client_negotiation.cc:769] Sending connection context
0806 04:58:10.851962 (+ 122us) client_negotiation.cc:240] Negotiation successful
0806 04:58:10.852089 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":355,"mutex_wait_us":63,"thread_start_us":469,"threads_started":1}
I20260806 04:58:10.852803 21051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:58:10.836624 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44900 (local address 127.9.96.66:41607)
0806 04:58:10.844445 (+ 7821us) server_negotiation.cc:206] Beginning negotiation
0806 04:58:10.844454 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 04:58:10.844475 (+ 21us) server_negotiation.cc:407] Connection header received
0806 04:58:10.844619 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:58:10.844625 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:58:10.844707 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:58:10.844854 (+ 147us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 04:58:10.845589 (+ 735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:10.846745 (+ 1156us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:10.849523 (+ 2778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:10.850692 (+ 1169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:10.852137 (+ 1445us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:58:10.852248 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0806 04:58:10.852301 (+ 53us) server_negotiation.cc:299] Negotiation successful
0806 04:58:10.852459 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7652,"thread_start_us":7001,"threads_started":1}
I20260806 04:58:10.853593 20804 tablet_service.cc:1467] Processing CreateTablet for tablet 49f6bbd66ffb43eba991b4548ba70f1f (DEFAULT_TABLE table=test-table [id=b417a4b0b88d483095249036e492c4d3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 04:58:10.853590 20574 tablet_service.cc:1467] Processing CreateTablet for tablet 49f6bbd66ffb43eba991b4548ba70f1f (DEFAULT_TABLE table=test-table [id=b417a4b0b88d483095249036e492c4d3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 04:58:10.854091 20804 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 49f6bbd66ffb43eba991b4548ba70f1f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 04:58:10.854094 20574 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 49f6bbd66ffb43eba991b4548ba70f1f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 04:58:10.855698 21057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:58:10.838849 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44068 (local address 127.9.96.65:37807)
0806 04:58:10.844238 (+ 5389us) server_negotiation.cc:206] Beginning negotiation
0806 04:58:10.844247 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 04:58:10.844272 (+ 25us) server_negotiation.cc:407] Connection header received
0806 04:58:10.844362 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:58:10.844368 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:58:10.844485 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:58:10.845474 (+ 989us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 04:58:10.845701 (+ 227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:10.846748 (+ 1047us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:10.849969 (+ 3221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:10.851202 (+ 1233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:10.851320 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:58:10.851434 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0806 04:58:10.852638 (+ 1204us) server_negotiation.cc:299] Negotiation successful
0806 04:58:10.852784 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5221,"thread_start_us":5056,"threads_started":1}
I20260806 04:58:10.859853 21059 tablet_bootstrap.cc:492] T 49f6bbd66ffb43eba991b4548ba70f1f P aab3b242cd3e451e958340a207168137: Bootstrap starting.
I20260806 04:58:10.859822 20348 tablet_service.cc:1467] Processing CreateTablet for tablet 49f6bbd66ffb43eba991b4548ba70f1f (DEFAULT_TABLE table=test-table [id=b417a4b0b88d483095249036e492c4d3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 04:58:10.860148 21058 tablet_bootstrap.cc:492] T 49f6bbd66ffb43eba991b4548ba70f1f P 50ea117b4ffa45bc8456eb9ec24594b3: Bootstrap starting.
I20260806 04:58:10.860268 20348 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 49f6bbd66ffb43eba991b4548ba70f1f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 04:58:10.861390 21058 tablet_bootstrap.cc:654] T 49f6bbd66ffb43eba991b4548ba70f1f P 50ea117b4ffa45bc8456eb9ec24594b3: Neither blocks nor log segments found. Creating new log.
I20260806 04:58:10.861572 21059 tablet_bootstrap.cc:654] T 49f6bbd66ffb43eba991b4548ba70f1f P aab3b242cd3e451e958340a207168137: Neither blocks nor log segments found. Creating new log.
I20260806 04:58:10.863508 21058 tablet_bootstrap.cc:492] T 49f6bbd66ffb43eba991b4548ba70f1f P 50ea117b4ffa45bc8456eb9ec24594b3: No bootstrap required, opened a new log
I20260806 04:58:10.863603 21059 tablet_bootstrap.cc:492] T 49f6bbd66ffb43eba991b4548ba70f1f P aab3b242cd3e451e958340a207168137: No bootstrap required, opened a new log
I20260806 04:58:10.863641 21058 ts_tablet_manager.cc:1397] T 49f6bbd66ffb43eba991b4548ba70f1f P 50ea117b4ffa45bc8456eb9ec24594b3: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260806 04:58:10.863700 21059 ts_tablet_manager.cc:1397] T 49f6bbd66ffb43eba991b4548ba70f1f P aab3b242cd3e451e958340a207168137: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260806 04:58:10.864265 21059 raft_consensus.cc:348] T 49f6bbd66ffb43eba991b4548ba70f1f P aab3b242cd3e451e958340a207168137 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aab3b242cd3e451e958340a207168137" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 38449 } } peers { permanent_uuid: "50ea117b4ffa45bc8456eb9ec24594b3" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 41607 } } peers { permanent_uuid: "2259df3ac9ca47bfb61228d968d524fd" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 37807 } }
I20260806 04:58:10.864454 21059 raft_consensus.cc:374] T 49f6bbd66ffb43eba991b4548ba70f1f P aab3b242cd3e451e958340a207168137 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 04:58:10.864517 21059 raft_consensus.cc:729] T 49f6bbd66ffb43eba991b4548ba70f1f P aab3b242cd3e451e958340a207168137 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aab3b242cd3e451e958340a207168137, State: Initialized, Role: FOLLOWER
I20260806 04:58:10.864441 21058 raft_consensus.cc:348] T 49f6bbd66ffb43eba991b4548ba70f1f P 50ea117b4ffa45bc8456eb9ec24594b3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aab3b242cd3e451e958340a207168137" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 38449 } } peers { permanent_uuid: "50ea117b4ffa45bc8456eb9ec24594b3" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 41607 } } peers { permanent_uuid: "2259df3ac9ca47bfb61228d968d524fd" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 37807 } }
I20260806 04:58:10.864679 21058 raft_consensus.cc:374] T 49f6bbd66ffb43eba991b4548ba70f1f P 50ea117b4ffa45bc8456eb9ec24594b3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 04:58:10.864763 21058 raft_consensus.cc:729] T 49f6bbd66ffb43eba991b4548ba70f1f P 50ea117b4ffa45bc8456eb9ec24594b3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 50ea117b4ffa45bc8456eb9ec24594b3, State: Initialized, Role: FOLLOWER
I20260806 04:58:10.864761 21059 consensus_queue.cc:260] T 49f6bbd66ffb43eba991b4548ba70f1f P aab3b242cd3e451e958340a207168137 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aab3b242cd3e451e958340a207168137" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 38449 } } peers { permanent_uuid: "50ea117b4ffa45bc8456eb9ec24594b3" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 41607 } } peers { permanent_uuid: "2259df3ac9ca47bfb61228d968d524fd" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 37807 } }
I20260806 04:58:10.864997 21058 consensus_queue.cc:260] T 49f6bbd66ffb43eba991b4548ba70f1f P 50ea117b4ffa45bc8456eb9ec24594b3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aab3b242cd3e451e958340a207168137" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 38449 } } peers { permanent_uuid: "50ea117b4ffa45bc8456eb9ec24594b3" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 41607 } } peers { permanent_uuid: "2259df3ac9ca47bfb61228d968d524fd" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 37807 } }
I20260806 04:58:10.865633 21059 ts_tablet_manager.cc:1428] T 49f6bbd66ffb43eba991b4548ba70f1f P aab3b242cd3e451e958340a207168137: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260806 04:58:10.867177 21031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:58:10.866764 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:40031 (local address 127.9.96.67:47947)
0806 04:58:10.866963 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0806 04:58:10.867060 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.96.126:40031: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
I20260806 04:58:10.868706 21031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:58:10.867976 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:40031 (local address 127.9.96.67:52935)
0806 04:58:10.868437 (+ 461us) negotiation.cc:107] Waiting for socket to connect
0806 04:58:10.868537 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.96.126:40031: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":349}
I20260806 04:58:10.871199 20902 heartbeater.cc:503] Master 127.9.96.125:39617 was elected leader, sending a full tablet report...
I20260806 04:58:10.873561 21067 tablet_bootstrap.cc:492] T 49f6bbd66ffb43eba991b4548ba70f1f P 2259df3ac9ca47bfb61228d968d524fd: Bootstrap starting.
I20260806 04:58:10.873867 21058 ts_tablet_manager.cc:1428] T 49f6bbd66ffb43eba991b4548ba70f1f P 50ea117b4ffa45bc8456eb9ec24594b3: Time spent starting tablet: real 0.010s user 0.006s sys 0.004s
I20260806 04:58:10.874465 21030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:58:10.874072 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:40031 (local address 127.9.96.66:44825)
0806 04:58:10.874210 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0806 04:58:10.874321 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.96.126:40031: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260806 04:58:10.875324 21030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:58:10.874998 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:40031 (local address 127.9.96.66:60215)
0806 04:58:10.875133 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0806 04:58:10.875195 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.96.126:40031: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260806 04:58:10.876444 20690 heartbeater.cc:503] Master 127.9.96.125:39617 was elected leader, sending a full tablet report...
I20260806 04:58:10.877708 21067 tablet_bootstrap.cc:654] T 49f6bbd66ffb43eba991b4548ba70f1f P 2259df3ac9ca47bfb61228d968d524fd: Neither blocks nor log segments found. Creating new log.
I20260806 04:58:10.879688 21067 tablet_bootstrap.cc:492] T 49f6bbd66ffb43eba991b4548ba70f1f P 2259df3ac9ca47bfb61228d968d524fd: No bootstrap required, opened a new log
I20260806 04:58:10.879853 21067 ts_tablet_manager.cc:1397] T 49f6bbd66ffb43eba991b4548ba70f1f P 2259df3ac9ca47bfb61228d968d524fd: Time spent bootstrapping tablet: real 0.006s user 0.003s sys 0.000s
I20260806 04:58:10.880478 21067 raft_consensus.cc:348] T 49f6bbd66ffb43eba991b4548ba70f1f P 2259df3ac9ca47bfb61228d968d524fd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aab3b242cd3e451e958340a207168137" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 38449 } } peers { permanent_uuid: "50ea117b4ffa45bc8456eb9ec24594b3" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 41607 } } peers { permanent_uuid: "2259df3ac9ca47bfb61228d968d524fd" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 37807 } }
I20260806 04:58:10.880681 21067 raft_consensus.cc:374] T 49f6bbd66ffb43eba991b4548ba70f1f P 2259df3ac9ca47bfb61228d968d524fd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 04:58:10.880797 21067 raft_consensus.cc:729] T 49f6bbd66ffb43eba991b4548ba70f1f P 2259df3ac9ca47bfb61228d968d524fd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2259df3ac9ca47bfb61228d968d524fd, State: Initialized, Role: FOLLOWER
I20260806 04:58:10.881031 21067 consensus_queue.cc:260] T 49f6bbd66ffb43eba991b4548ba70f1f P 2259df3ac9ca47bfb61228d968d524fd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aab3b242cd3e451e958340a207168137" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 38449 } } peers { permanent_uuid: "50ea117b4ffa45bc8456eb9ec24594b3" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 41607 } } peers { permanent_uuid: "2259df3ac9ca47bfb61228d968d524fd" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 37807 } }
I20260806 04:58:10.881790 21067 ts_tablet_manager.cc:1428] T 49f6bbd66ffb43eba991b4548ba70f1f P 2259df3ac9ca47bfb61228d968d524fd: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260806 04:58:10.882875 21032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:58:10.882466 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:40031 (local address 127.9.96.65:59673)
0806 04:58:10.882654 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0806 04:58:10.882741 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.96.126:40031: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
I20260806 04:58:10.884097 20447 heartbeater.cc:503] Master 127.9.96.125:39617 was elected leader, sending a full tablet report...
I20260806 04:58:10.884846 21032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:58:10.884368 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:40031 (local address 127.9.96.65:40141)
0806 04:58:10.884574 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0806 04:58:10.884683 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.96.126:40031: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99}
I20260806 04:58:10.888331 21069 raft_consensus.cc:482] T 49f6bbd66ffb43eba991b4548ba70f1f P 2259df3ac9ca47bfb61228d968d524fd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 04:58:10.888537 21069 raft_consensus.cc:504] T 49f6bbd66ffb43eba991b4548ba70f1f P 2259df3ac9ca47bfb61228d968d524fd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aab3b242cd3e451e958340a207168137" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 38449 } } peers { permanent_uuid: "50ea117b4ffa45bc8456eb9ec24594b3" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 41607 } } peers { permanent_uuid: "2259df3ac9ca47bfb61228d968d524fd" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 37807 } }
I20260806 04:58:10.889155 21069 leader_election.cc:288] T 49f6bbd66ffb43eba991b4548ba70f1f P 2259df3ac9ca47bfb61228d968d524fd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers aab3b242cd3e451e958340a207168137 (127.9.96.67:38449), 50ea117b4ffa45bc8456eb9ec24594b3 (127.9.96.66:41607)
I20260806 04:58:10.889631 20840 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "49f6bbd66ffb43eba991b4548ba70f1f" candidate_uuid: "2259df3ac9ca47bfb61228d968d524fd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aab3b242cd3e451e958340a207168137" is_pre_election: true
I20260806 04:58:10.889700 20609 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "49f6bbd66ffb43eba991b4548ba70f1f" candidate_uuid: "2259df3ac9ca47bfb61228d968d524fd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "50ea117b4ffa45bc8456eb9ec24594b3" is_pre_election: true
I20260806 04:58:10.889863 20840 raft_consensus.cc:2454] T 49f6bbd66ffb43eba991b4548ba70f1f P aab3b242cd3e451e958340a207168137 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2259df3ac9ca47bfb61228d968d524fd in term 0.
I20260806 04:58:10.889973 20609 raft_consensus.cc:2454] T 49f6bbd66ffb43eba991b4548ba70f1f P 50ea117b4ffa45bc8456eb9ec24594b3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2259df3ac9ca47bfb61228d968d524fd in term 0.
I20260806 04:58:10.890360 20289 leader_election.cc:302] T 49f6bbd66ffb43eba991b4548ba70f1f P 2259df3ac9ca47bfb61228d968d524fd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2259df3ac9ca47bfb61228d968d524fd, aab3b242cd3e451e958340a207168137; no voters:
I20260806 04:58:10.890630 21069 raft_consensus.cc:2781] T 49f6bbd66ffb43eba991b4548ba70f1f P 2259df3ac9ca47bfb61228d968d524fd [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 04:58:10.890763 21069 raft_consensus.cc:482] T 49f6bbd66ffb43eba991b4548ba70f1f P 2259df3ac9ca47bfb61228d968d524fd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 04:58:10.890872 21069 raft_consensus.cc:3037] T 49f6bbd66ffb43eba991b4548ba70f1f P 2259df3ac9ca47bfb61228d968d524fd [term 0 FOLLOWER]: Advancing to term 1
I20260806 04:58:10.892931 21069 raft_consensus.cc:504] T 49f6bbd66ffb43eba991b4548ba70f1f P 2259df3ac9ca47bfb61228d968d524fd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aab3b242cd3e451e958340a207168137" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 38449 } } peers { permanent_uuid: "50ea117b4ffa45bc8456eb9ec24594b3" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 41607 } } peers { permanent_uuid: "2259df3ac9ca47bfb61228d968d524fd" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 37807 } }
I20260806 04:58:10.893518 21069 leader_election.cc:288] T 49f6bbd66ffb43eba991b4548ba70f1f P 2259df3ac9ca47bfb61228d968d524fd [CANDIDATE]: Term 1 election: Requested vote from peers aab3b242cd3e451e958340a207168137 (127.9.96.67:38449), 50ea117b4ffa45bc8456eb9ec24594b3 (127.9.96.66:41607)
I20260806 04:58:10.894029 20840 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "49f6bbd66ffb43eba991b4548ba70f1f" candidate_uuid: "2259df3ac9ca47bfb61228d968d524fd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aab3b242cd3e451e958340a207168137"
I20260806 04:58:10.894120 20609 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "49f6bbd66ffb43eba991b4548ba70f1f" candidate_uuid: "2259df3ac9ca47bfb61228d968d524fd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "50ea117b4ffa45bc8456eb9ec24594b3"
I20260806 04:58:10.894245 20840 raft_consensus.cc:3037] T 49f6bbd66ffb43eba991b4548ba70f1f P aab3b242cd3e451e958340a207168137 [term 0 FOLLOWER]: Advancing to term 1
I20260806 04:58:10.894312 20609 raft_consensus.cc:3037] T 49f6bbd66ffb43eba991b4548ba70f1f P 50ea117b4ffa45bc8456eb9ec24594b3 [term 0 FOLLOWER]: Advancing to term 1
I20260806 04:58:10.897360 20840 raft_consensus.cc:2454] T 49f6bbd66ffb43eba991b4548ba70f1f P aab3b242cd3e451e958340a207168137 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2259df3ac9ca47bfb61228d968d524fd in term 1.
I20260806 04:58:10.897363 20609 raft_consensus.cc:2454] T 49f6bbd66ffb43eba991b4548ba70f1f P 50ea117b4ffa45bc8456eb9ec24594b3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2259df3ac9ca47bfb61228d968d524fd in term 1.
I20260806 04:58:10.897760 20289 leader_election.cc:302] T 49f6bbd66ffb43eba991b4548ba70f1f P 2259df3ac9ca47bfb61228d968d524fd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2259df3ac9ca47bfb61228d968d524fd, 50ea117b4ffa45bc8456eb9ec24594b3; no voters:
I20260806 04:58:10.898029 21069 raft_consensus.cc:2781] T 49f6bbd66ffb43eba991b4548ba70f1f P 2259df3ac9ca47bfb61228d968d524fd [term 1 FOLLOWER]: Leader election won for term 1
I20260806 04:58:10.898461 21069 raft_consensus.cc:686] T 49f6bbd66ffb43eba991b4548ba70f1f P 2259df3ac9ca47bfb61228d968d524fd [term 1 LEADER]: Becoming Leader. State: Replica: 2259df3ac9ca47bfb61228d968d524fd, State: Running, Role: LEADER
I20260806 04:58:10.898743 21069 consensus_queue.cc:237] T 49f6bbd66ffb43eba991b4548ba70f1f P 2259df3ac9ca47bfb61228d968d524fd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aab3b242cd3e451e958340a207168137" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 38449 } } peers { permanent_uuid: "50ea117b4ffa45bc8456eb9ec24594b3" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 41607 } } peers { permanent_uuid: "2259df3ac9ca47bfb61228d968d524fd" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 37807 } }
I20260806 04:58:10.899533 21032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:58:10.899017 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:40031 (local address 127.9.96.65:47051)
0806 04:58:10.899204 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0806 04:58:10.899321 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.96.126:40031: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260806 04:58:10.901558 19866 catalog_manager.cc:5495] T 49f6bbd66ffb43eba991b4548ba70f1f P 2259df3ac9ca47bfb61228d968d524fd reported cstate change: term changed from 0 to 1, leader changed from <none> to 2259df3ac9ca47bfb61228d968d524fd (127.9.96.65). New cstate: current_term: 1 leader_uuid: "2259df3ac9ca47bfb61228d968d524fd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aab3b242cd3e451e958340a207168137" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 38449 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "50ea117b4ffa45bc8456eb9ec24594b3" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 41607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2259df3ac9ca47bfb61228d968d524fd" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 37807 } health_report { overall_health: HEALTHY } } }
I20260806 04:58:10.926457 21057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:58:10.922119 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44072 (local address 127.9.96.65:37807)
0806 04:58:10.922297 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0806 04:58:10.922301 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 04:58:10.922318 (+ 17us) server_negotiation.cc:407] Connection header received
0806 04:58:10.922448 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:58:10.922451 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:58:10.922514 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:58:10.922588 (+ 74us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 04:58:10.922905 (+ 317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:10.923730 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:10.924983 (+ 1253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:10.925404 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:10.925496 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:58:10.925894 (+ 398us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 04:58:10.925992 (+ 98us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 04:58:10.926059 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0806 04:58:10.926195 (+ 136us) server_negotiation.cc:299] Negotiation successful
0806 04:58:10.926257 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260806 04:58:10.929502 20609 raft_consensus.cc:1261] T 49f6bbd66ffb43eba991b4548ba70f1f P 50ea117b4ffa45bc8456eb9ec24594b3 [term 1 FOLLOWER]: Refusing update from remote peer 2259df3ac9ca47bfb61228d968d524fd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 04:58:10.929572 20840 raft_consensus.cc:1261] T 49f6bbd66ffb43eba991b4548ba70f1f P aab3b242cd3e451e958340a207168137 [term 1 FOLLOWER]: Refusing update from remote peer 2259df3ac9ca47bfb61228d968d524fd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 04:58:10.930148 21070 consensus_queue.cc:1035] T 49f6bbd66ffb43eba991b4548ba70f1f P 2259df3ac9ca47bfb61228d968d524fd [LEADER]: Connected to new peer: Peer: permanent_uuid: "50ea117b4ffa45bc8456eb9ec24594b3" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 41607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 04:58:10.930526 21069 consensus_queue.cc:1035] T 49f6bbd66ffb43eba991b4548ba70f1f P 2259df3ac9ca47bfb61228d968d524fd [LEADER]: Connected to new peer: Peer: permanent_uuid: "aab3b242cd3e451e958340a207168137" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 38449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 04:58:10.932947 21030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:58:10.932444 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:40031 (local address 127.9.96.66:47403)
0806 04:58:10.932690 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0806 04:58:10.932793 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.96.126:40031: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":78}
I20260806 04:58:10.932973 21031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:58:10.932302 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:40031 (local address 127.9.96.67:58189)
0806 04:58:10.932622 (+ 320us) negotiation.cc:107] Waiting for socket to connect
0806 04:58:10.932769 (+ 147us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.96.126:40031: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":164}
I20260806 04:58:10.936647 21076 mvcc.cc:205] Tried to move back new op lower bound from 7315424423641153536 to 7315424423524433920. Current Snapshot: MvccSnapshot[applied={T|T < 7315424423641153536}]
I20260806 04:58:10.937983 21031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:58:10.937326 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:40031 (local address 127.9.96.67:58275)
0806 04:58:10.937616 (+ 290us) negotiation.cc:107] Waiting for socket to connect
0806 04:58:10.937744 (+ 128us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.96.126:40031: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":138}
I20260806 04:58:10.943106 21075 mvcc.cc:205] Tried to move back new op lower bound from 7315424423641153536 to 7315424423524433920. Current Snapshot: MvccSnapshot[applied={T|T < 7315424423641153536}]
I20260806 04:58:10.947247 21032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:58:10.946623 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:40031 (local address 127.9.96.65:60319)
0806 04:58:10.946943 (+ 320us) negotiation.cc:107] Waiting for socket to connect
0806 04:58:10.947046 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.96.126:40031: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":203}
I20260806 04:58:10.948962 21030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:58:10.948371 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:40031 (local address 127.9.96.66:52791)
0806 04:58:10.948665 (+ 294us) negotiation.cc:107] Waiting for socket to connect
0806 04:58:10.948750 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.96.126:40031: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":198}
I20260806 04:58:10.950680 21032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:58:10.950236 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:40031 (local address 127.9.96.65:54523)
0806 04:58:10.950433 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0806 04:58:10.950541 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.96.126:40031: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":92}
I20260806 04:58:10.952260 19865 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53044:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 04:58:10.952550 19865 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53044:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 04:58:10.958194 19865 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 89e9df10c70d4ddaacd2a30fff7a3b2d: Sending DeleteTablet for 3 replicas of tablet 49f6bbd66ffb43eba991b4548ba70f1f
I20260806 04:58:10.959278 20804 tablet_service.cc:1514] Processing DeleteTablet for tablet 49f6bbd66ffb43eba991b4548ba70f1f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 04:58:10 UTC) from {username='oryx', principal='oryx/127.9.96.125@KRBTEST.COM'} at 127.0.0.1:58126
I20260806 04:58:10.959411 20574 tablet_service.cc:1514] Processing DeleteTablet for tablet 49f6bbd66ffb43eba991b4548ba70f1f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 04:58:10 UTC) from {username='oryx', principal='oryx/127.9.96.125@KRBTEST.COM'} at 127.0.0.1:44900
I20260806 04:58:10.959656 20348 tablet_service.cc:1514] Processing DeleteTablet for tablet 49f6bbd66ffb43eba991b4548ba70f1f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 04:58:10 UTC) from {username='oryx', principal='oryx/127.9.96.125@KRBTEST.COM'} at 127.0.0.1:44068
I20260806 04:58:10.960987 9601 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu pid 20161
I20260806 04:58:10.964180 21092 tablet_replica.cc:330] T 49f6bbd66ffb43eba991b4548ba70f1f P 50ea117b4ffa45bc8456eb9ec24594b3: stopping tablet replica
I20260806 04:58:10.964514 21092 raft_consensus.cc:2229] T 49f6bbd66ffb43eba991b4548ba70f1f P 50ea117b4ffa45bc8456eb9ec24594b3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 04:58:10.965015 21092 raft_consensus.cc:2258] T 49f6bbd66ffb43eba991b4548ba70f1f P 50ea117b4ffa45bc8456eb9ec24594b3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 04:58:10.966213 21093 tablet_replica.cc:330] T 49f6bbd66ffb43eba991b4548ba70f1f P 2259df3ac9ca47bfb61228d968d524fd: stopping tablet replica
I20260806 04:58:10.966464 21093 raft_consensus.cc:2229] T 49f6bbd66ffb43eba991b4548ba70f1f P 2259df3ac9ca47bfb61228d968d524fd [term 1 LEADER]: Raft consensus shutting down.
I20260806 04:58:10.966684 21092 ts_tablet_manager.cc:1905] T 49f6bbd66ffb43eba991b4548ba70f1f P 50ea117b4ffa45bc8456eb9ec24594b3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 04:58:10.966969 21093 raft_consensus.cc:2258] T 49f6bbd66ffb43eba991b4548ba70f1f P 2259df3ac9ca47bfb61228d968d524fd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 04:58:10.967681 21091 tablet_replica.cc:330] T 49f6bbd66ffb43eba991b4548ba70f1f P aab3b242cd3e451e958340a207168137: stopping tablet replica
I20260806 04:58:10.967895 21091 raft_consensus.cc:2229] T 49f6bbd66ffb43eba991b4548ba70f1f P aab3b242cd3e451e958340a207168137 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 04:58:10.968348 21091 raft_consensus.cc:2258] T 49f6bbd66ffb43eba991b4548ba70f1f P aab3b242cd3e451e958340a207168137 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 04:58:10.968967 21093 ts_tablet_manager.cc:1905] T 49f6bbd66ffb43eba991b4548ba70f1f P 2259df3ac9ca47bfb61228d968d524fd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 04:58:10.974182 21092 ts_tablet_manager.cc:1918] T 49f6bbd66ffb43eba991b4548ba70f1f P 50ea117b4ffa45bc8456eb9ec24594b3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 04:58:10.974336 21092 log.cc:1196] T 49f6bbd66ffb43eba991b4548ba70f1f P 50ea117b4ffa45bc8456eb9ec24594b3: Deleting WAL directory at /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785992227050689-9601-0/minicluster-data/ts-1/wal/wals/49f6bbd66ffb43eba991b4548ba70f1f
I20260806 04:58:10.974879 21092 ts_tablet_manager.cc:1939] T 49f6bbd66ffb43eba991b4548ba70f1f P 50ea117b4ffa45bc8456eb9ec24594b3: Deleting consensus metadata
I20260806 04:58:10.975266 21093 ts_tablet_manager.cc:1918] T 49f6bbd66ffb43eba991b4548ba70f1f P 2259df3ac9ca47bfb61228d968d524fd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 04:58:10.975442 21093 log.cc:1196] T 49f6bbd66ffb43eba991b4548ba70f1f P 2259df3ac9ca47bfb61228d968d524fd: Deleting WAL directory at /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785992227050689-9601-0/minicluster-data/ts-0/wal/wals/49f6bbd66ffb43eba991b4548ba70f1f
I20260806 04:58:10.975835 21093 ts_tablet_manager.cc:1939] T 49f6bbd66ffb43eba991b4548ba70f1f P 2259df3ac9ca47bfb61228d968d524fd: Deleting consensus metadata
I20260806 04:58:10.976171 19847 catalog_manager.cc:4841] TS 50ea117b4ffa45bc8456eb9ec24594b3 (127.9.96.66:41607): tablet 49f6bbd66ffb43eba991b4548ba70f1f (table test-table [id=b417a4b0b88d483095249036e492c4d3]) successfully deleted
I20260806 04:58:10.976791 21091 ts_tablet_manager.cc:1905] T 49f6bbd66ffb43eba991b4548ba70f1f P aab3b242cd3e451e958340a207168137: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 04:58:10.976863 19847 catalog_manager.cc:4841] TS 2259df3ac9ca47bfb61228d968d524fd (127.9.96.65:37807): tablet 49f6bbd66ffb43eba991b4548ba70f1f (table test-table [id=b417a4b0b88d483095249036e492c4d3]) successfully deleted
I20260806 04:58:10.982288 21091 ts_tablet_manager.cc:1918] T 49f6bbd66ffb43eba991b4548ba70f1f P aab3b242cd3e451e958340a207168137: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 04:58:10.982447 21091 log.cc:1196] T 49f6bbd66ffb43eba991b4548ba70f1f P aab3b242cd3e451e958340a207168137: Deleting WAL directory at /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785992227050689-9601-0/minicluster-data/ts-2/wal/wals/49f6bbd66ffb43eba991b4548ba70f1f
I20260806 04:58:10.982861 21091 ts_tablet_manager.cc:1939] T 49f6bbd66ffb43eba991b4548ba70f1f P aab3b242cd3e451e958340a207168137: Deleting consensus metadata
I20260806 04:58:10.984009 21057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:58:10.961805 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44082 (local address 127.9.96.65:37807)
0806 04:58:10.961974 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0806 04:58:10.961981 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 04:58:10.965805 (+ 3824us) server_negotiation.cc:407] Connection header received
0806 04:58:10.967685 (+ 1880us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:58:10.967691 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:58:10.967774 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:58:10.967882 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:58:10.969832 (+ 1950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:10.970945 (+ 1113us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:10.972936 (+ 1991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:10.973670 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:10.973813 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:58:10.978058 (+ 4245us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:58:10.978078 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:58:10.978103 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:58:10.978148 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:58:10.981113 (+ 2965us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:58:10.981887 (+ 774us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:58:10.981893 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:58:10.981896 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:58:10.982018 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:58:10.982456 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:58:10.982462 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:58:10.982465 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:58:10.982860 (+ 395us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:58:10.983136 (+ 276us) server_negotiation.cc:1035] Waiting for connection context
0806 04:58:10.983528 (+ 392us) server_negotiation.cc:299] Negotiation successful
0806 04:58:10.983692 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260806 04:58:10.984223 19847 catalog_manager.cc:4841] TS aab3b242cd3e451e958340a207168137 (127.9.96.67:38449): tablet 49f6bbd66ffb43eba991b4548ba70f1f (table test-table [id=b417a4b0b88d483095249036e492c4d3]) successfully deleted
I20260806 04:58:10.984721 20434 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 04:58:11.104180 9601 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu with pid 20161
I20260806 04:58:11.134991 9601 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu pid 20466
I20260806 04:58:11.148718 21051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:58:11.135774 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44914 (local address 127.9.96.66:41607)
0806 04:58:11.136006 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0806 04:58:11.136014 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 04:58:11.136035 (+ 21us) server_negotiation.cc:407] Connection header received
0806 04:58:11.136239 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:58:11.136244 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:58:11.136316 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:58:11.136423 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:58:11.138039 (+ 1616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:11.138722 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:11.140049 (+ 1327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:11.140485 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:11.140633 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:58:11.143074 (+ 2441us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:58:11.143097 (+ 23us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:58:11.143114 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:58:11.143148 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:58:11.145877 (+ 2729us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:58:11.146517 (+ 640us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:58:11.146533 (+ 16us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:58:11.146535 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:58:11.146673 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:58:11.147089 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:58:11.147094 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:58:11.147097 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:58:11.147530 (+ 433us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:58:11.147868 (+ 338us) server_negotiation.cc:1035] Waiting for connection context
0806 04:58:11.148145 (+ 277us) server_negotiation.cc:299] Negotiation successful
0806 04:58:11.148380 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260806 04:58:11.149415 20672 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 04:58:11.241341 9601 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu with pid 20466
I20260806 04:58:11.267037 9601 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu pid 20704
I20260806 04:58:11.280207 21048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:58:11.267910 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58134 (local address 127.9.96.67:38449)
0806 04:58:11.268140 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0806 04:58:11.268145 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 04:58:11.268167 (+ 22us) server_negotiation.cc:407] Connection header received
0806 04:58:11.268342 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:58:11.268346 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:58:11.268411 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:58:11.268494 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:58:11.269875 (+ 1381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:11.270587 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:11.271973 (+ 1386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:11.272407 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:11.272496 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:58:11.274859 (+ 2363us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:58:11.274873 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:58:11.274890 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:58:11.274937 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:58:11.277836 (+ 2899us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:58:11.278387 (+ 551us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:58:11.278395 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:58:11.278397 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:58:11.278536 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:58:11.278847 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:58:11.278850 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:58:11.278852 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:58:11.279172 (+ 320us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:58:11.279474 (+ 302us) server_negotiation.cc:1035] Waiting for connection context
0806 04:58:11.279708 (+ 234us) server_negotiation.cc:299] Negotiation successful
0806 04:58:11.279880 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260806 04:58:11.280895 20894 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 04:58:11.333382 21054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:58:11.332961 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:40031 (local address 127.0.0.1:59872)
0806 04:58:11.333146 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0806 04:58:11.333229 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.96.126:40031: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260806 04:58:11.334719 21054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:58:11.334383 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:40031 (local address 127.0.0.1:59874)
0806 04:58:11.334531 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0806 04:58:11.334618 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.96.126:40031: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260806 04:58:11.382149 9601 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu with pid 20704
I20260806 04:58:11.405098 9601 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu pid 19640
I20260806 04:58:11.406251 19916 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 04:58:11.531401 9601 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu with pid 19640
W20260806 04:58:11.561110 20157 connection.cc:535] server connection from 127.0.0.1:55218 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260806 04:58:11.561421 9601 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu pid 19806
I20260806 04:58:11.562496 20223 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 04:58:11.676258 9601 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu with pid 19806
2026-08-06T04:58:11Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (11485 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785992227050689-9601-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 04:58:11 dist-test-slave-x4qh krb5kdc[21106](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 04:58:11 dist-test-slave-x4qh krb5kdc[21106](info): set up 2 sockets
Aug 06 04:58:11 dist-test-slave-x4qh krb5kdc[21106](info): commencing operation
krb5kdc: starting...
W20260806 04:58:12.964416 9601 mini_kdc.cc:121] Time spent starting KDC: real 1.237s user 0.008s sys 0.020s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785992227050689-9601-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785992227050689-9601-0/krb5kdc/test-user.keytab.
Aug 06 04:58:13 dist-test-slave-x4qh krb5kdc[21106](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992293, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T04:58:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T04:58:13Z Disabled control of system clock
WARNING: no policy specified for oryx/127.9.96.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.9.96.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.9.96.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785992227050689-9601-0/krb5kdc/oryx_127.9.96.126.keytab.
Entry for principal oryx/127.9.96.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785992227050689-9601-0/krb5kdc/oryx_127.9.96.126.keytab.
I20260806 04:58:13.190472 9601 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
/tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785992227050689-9601-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785992227050689-9601-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785992227050689-9601-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785992227050689-9601-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.96.126:40621
--webserver_interface=127.9.96.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785992227050689-9601-0/krb5kdc/oryx_127.9.96.126.keytab
--principal=oryx/127.9.96.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.96.84:46011
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.96.126:40621
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785992227050689-9601-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785992227050689-9601-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785992227050689-9601-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 04:58:13.511914 21124 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 04:58:13.512212 21124 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 04:58:13.512451 21124 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 04:58:13.520924 21124 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 04:58:13.521035 21124 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260806 04:58:13.521107 21124 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 04:58:13.521193 21124 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 04:58:13.521278 21124 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 04:58:13.521358 21124 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 04:58:13.531915 21124 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.96.84:46011
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785992227050689-9601-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785992227050689-9601-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.96.126:40621
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785992227050689-9601-0/krb5kdc/oryx_127.9.96.126.keytab
--principal=oryx/127.9.96.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.96.126:40621
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785992227050689-9601-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.96.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785992227050689-9601-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 03:56:12 UTC on ccycloud.quasar-guijbm.root.comops.site
build id 4569618
ASAN enabled
I20260806 04:58:13.533129 21124 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 04:58:13.534416 21124 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 04:58:13.546169 21133 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:58:13.546183 21131 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:58:13.546452 21130 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:58:13.546905 21124 server_base.cc:1034] running on GCE node
I20260806 04:58:13.547740 21124 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 04:58:13.549036 21124 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 04:58:13.550257 21124 hybrid_clock.cc:648] HybridClock initialized: now 1785992293550202 us; error 26 us; skew 500 ppm
I20260806 04:58:13.550577 21124 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 04:58:13 dist-test-slave-x4qh krb5kdc[21106](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992293, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.96.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 04:58:13.553843 21124 init.cc:377] Logged in from keytab as oryx/127.9.96.126@KRBTEST.COM (short username oryx)
I20260806 04:58:13.555148 21124 webserver.cc:460] Webserver started at http://127.9.96.126:34099/ using document root <none> and password file <none>
I20260806 04:58:13.555872 21124 fs_manager.cc:359] Metadata directory not provided
I20260806 04:58:13.555998 21124 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 04:58:13.556329 21124 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 04:58:13.559072 21124 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785992227050689-9601-0/minicluster-data/master-0/data/instance:
uuid: "6015b5a9ea39484baf36a8765add32e3"
format_stamp: "Formatted at 2026-08-06 04:58:13 on dist-test-slave-x4qh"
server_key: "21ad2902c7dd1c9bef163feb6d51024b"
server_key_iv: "3a27f6dd82be471faf85da088e56508e"
server_key_version: "encryptionkey@0"
I20260806 04:58:13.559890 21124 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785992227050689-9601-0/minicluster-data/master-0/wal/instance:
uuid: "6015b5a9ea39484baf36a8765add32e3"
format_stamp: "Formatted at 2026-08-06 04:58:13 on dist-test-slave-x4qh"
server_key: "21ad2902c7dd1c9bef163feb6d51024b"
server_key_iv: "3a27f6dd82be471faf85da088e56508e"
server_key_version: "encryptionkey@0"
I20260806 04:58:13.564599 21124 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260806 04:58:13.567778 21145 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 04:58:13.568899 21124 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.002s
I20260806 04:58:13.569079 21124 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785992227050689-9601-0/minicluster-data/master-0/data,/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785992227050689-9601-0/minicluster-data/master-0/wal
uuid: "6015b5a9ea39484baf36a8765add32e3"
format_stamp: "Formatted at 2026-08-06 04:58:13 on dist-test-slave-x4qh"
server_key: "21ad2902c7dd1c9bef163feb6d51024b"
server_key_iv: "3a27f6dd82be471faf85da088e56508e"
server_key_version: "encryptionkey@0"
I20260806 04:58:13.569289 21124 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785992227050689-9601-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785992227050689-9601-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785992227050689-9601-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 04:58:13.581430 21124 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 04:58:13.585724 21124 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 04:58:13.586009 21124 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 04:58:13.606515 21124 rpc_server.cc:307] RPC server started. Bound to: 127.9.96.126:40621
I20260806 04:58:13.606532 21228 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.96.126:40621 every 8 connection(s)
I20260806 04:58:13.607867 21124 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785992227050689-9601-0/minicluster-data/master-0/data/info.pb
I20260806 04:58:13.611796 21229 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 04:58:13.614174 9601 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu as pid 21124
I20260806 04:58:13.614423 9601 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785992227050689-9601-0/minicluster-data/master-0/wal/instance
I20260806 04:58:13.615105 9601 external_mini_cluster.cc:1428] Setting key 0b870328edf736b1c53c15c1477b2861
I20260806 04:58:13.622843 21229 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6015b5a9ea39484baf36a8765add32e3: Bootstrap starting.
I20260806 04:58:13.625841 21229 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6015b5a9ea39484baf36a8765add32e3: Neither blocks nor log segments found. Creating new log.
I20260806 04:58:13.626809 21229 log.cc:824] T 00000000000000000000000000000000 P 6015b5a9ea39484baf36a8765add32e3: Log is configured to *not* fsync() on all Append() calls
Aug 06 04:58:13 dist-test-slave-x4qh krb5kdc[21106](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992293, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.9.96.126@KRBTEST.COM
I20260806 04:58:13.630064 21229 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6015b5a9ea39484baf36a8765add32e3: No bootstrap required, opened a new log
I20260806 04:58:13.637516 21234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:58:13.617203 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57192 (local address 127.9.96.126:40621)
0806 04:58:13.617957 (+ 754us) server_negotiation.cc:206] Beginning negotiation
0806 04:58:13.617968 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 04:58:13.618009 (+ 41us) server_negotiation.cc:407] Connection header received
0806 04:58:13.619705 (+ 1696us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:58:13.619731 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:58:13.620085 (+ 354us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:58:13.620467 (+ 382us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:58:13.621865 (+ 1398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:13.623193 (+ 1328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:13.624497 (+ 1304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:13.625142 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:13.625342 (+ 200us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:58:13.630277 (+ 4935us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:58:13.630284 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:58:13.630304 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:58:13.630341 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:58:13.633537 (+ 3196us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:58:13.634364 (+ 827us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:58:13.634370 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:58:13.634375 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:58:13.634557 (+ 182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:58:13.634967 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:58:13.634970 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:58:13.634972 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:58:13.635535 (+ 563us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:58:13.635752 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0806 04:58:13.636171 (+ 419us) server_negotiation.cc:299] Negotiation successful
0806 04:58:13.636551 (+ 380us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":565,"thread_start_us":360,"threads_started":1}
I20260806 04:58:13.638760 21229 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6015b5a9ea39484baf36a8765add32e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6015b5a9ea39484baf36a8765add32e3" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 40621 } }
I20260806 04:58:13.639111 21229 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6015b5a9ea39484baf36a8765add32e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 04:58:13.639240 21229 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6015b5a9ea39484baf36a8765add32e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6015b5a9ea39484baf36a8765add32e3, State: Initialized, Role: FOLLOWER
I20260806 04:58:13.639816 21229 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6015b5a9ea39484baf36a8765add32e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6015b5a9ea39484baf36a8765add32e3" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 40621 } }
I20260806 04:58:13.640108 21229 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6015b5a9ea39484baf36a8765add32e3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 04:58:13.640282 21229 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6015b5a9ea39484baf36a8765add32e3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 04:58:13.640471 21229 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6015b5a9ea39484baf36a8765add32e3 [term 0 FOLLOWER]: Advancing to term 1
I20260806 04:58:13.642436 21229 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6015b5a9ea39484baf36a8765add32e3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6015b5a9ea39484baf36a8765add32e3" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 40621 } }
I20260806 04:58:13.642846 21229 leader_election.cc:302] T 00000000000000000000000000000000 P 6015b5a9ea39484baf36a8765add32e3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6015b5a9ea39484baf36a8765add32e3; no voters:
I20260806 04:58:13.643316 21229 leader_election.cc:288] T 00000000000000000000000000000000 P 6015b5a9ea39484baf36a8765add32e3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 04:58:13.643503 21236 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6015b5a9ea39484baf36a8765add32e3 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 04:58:13.644248 21236 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6015b5a9ea39484baf36a8765add32e3 [term 1 LEADER]: Becoming Leader. State: Replica: 6015b5a9ea39484baf36a8765add32e3, State: Running, Role: LEADER
I20260806 04:58:13.644754 21236 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6015b5a9ea39484baf36a8765add32e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6015b5a9ea39484baf36a8765add32e3" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 40621 } }
I20260806 04:58:13.645675 21229 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6015b5a9ea39484baf36a8765add32e3 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 04:58:13.647830 21238 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6015b5a9ea39484baf36a8765add32e3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6015b5a9ea39484baf36a8765add32e3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6015b5a9ea39484baf36a8765add32e3" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 40621 } } }
I20260806 04:58:13.648346 21238 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6015b5a9ea39484baf36a8765add32e3 [sys.catalog]: This master's current role is: LEADER
I20260806 04:58:13.649148 21241 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6015b5a9ea39484baf36a8765add32e3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6015b5a9ea39484baf36a8765add32e3. Latest consensus state: current_term: 1 leader_uuid: "6015b5a9ea39484baf36a8765add32e3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6015b5a9ea39484baf36a8765add32e3" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 40621 } } }
I20260806 04:58:13.649485 21241 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6015b5a9ea39484baf36a8765add32e3 [sys.catalog]: This master's current role is: LEADER
I20260806 04:58:13.653879 21249 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 04:58:13.657991 21249 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 04:58:13.665448 21249 catalog_manager.cc:1347] Generated new cluster ID: 7f97665020044f22804169159a198148
I20260806 04:58:13.665565 21249 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 04:58:13.672554 21249 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 04:58:13.674371 21249 catalog_manager.cc:1504] Loading token signing keys...
I20260806 04:58:13.684657 21249 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6015b5a9ea39484baf36a8765add32e3: Generated new TSK 0
I20260806 04:58:13.685568 21249 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.9.96.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.9.96.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.9.96.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785992227050689-9601-0/krb5kdc/oryx_127.9.96.65.keytab.
Entry for principal oryx/127.9.96.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785992227050689-9601-0/krb5kdc/oryx_127.9.96.65.keytab.
I20260806 04:58:13.748629 9601 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
/tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785992227050689-9601-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785992227050689-9601-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785992227050689-9601-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785992227050689-9601-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.96.65:0
--local_ip_for_outbound_sockets=127.9.96.65
--webserver_interface=127.9.96.65
--webserver_port=0
--tserver_master_addrs=127.9.96.126:40621
--keytab_file=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785992227050689-9601-0/krb5kdc/oryx_127.9.96.65.keytab
--principal=oryx/127.9.96.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.96.84:46011
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785992227050689-9601-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785992227050689-9601-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785992227050689-9601-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 04:58:14.081712 21270 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 04:58:14.082056 21270 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 04:58:14.082242 21270 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 04:58:14.089524 21270 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 04:58:14.089706 21270 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 04:58:14.089864 21270 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 04:58:14.090018 21270 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.96.65
I20260806 04:58:14.101269 21270 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.96.84:46011
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785992227050689-9601-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785992227050689-9601-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785992227050689-9601-0/krb5kdc/oryx_127.9.96.65.keytab
--principal=oryx/127.9.96.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.96.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785992227050689-9601-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.96.65
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.9.96.126:40621
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.96.65
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785992227050689-9601-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 03:56:12 UTC on ccycloud.quasar-guijbm.root.comops.site
build id 4569618
ASAN enabled
I20260806 04:58:14.102728 21270 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 04:58:14.104233 21270 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 04:58:14.117708 21277 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:58:14.117717 21281 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:58:14.117687 21278 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:58:14.118846 21270 server_base.cc:1034] running on GCE node
I20260806 04:58:14.119624 21270 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 04:58:14.120630 21270 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 04:58:14.121870 21270 hybrid_clock.cc:648] HybridClock initialized: now 1785992294121790 us; error 52 us; skew 500 ppm
I20260806 04:58:14.122202 21270 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 04:58:14 dist-test-slave-x4qh krb5kdc[21106](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992294, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.96.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 04:58:14.125862 21270 init.cc:377] Logged in from keytab as oryx/127.9.96.65@KRBTEST.COM (short username oryx)
I20260806 04:58:14.127424 21270 webserver.cc:460] Webserver started at http://127.9.96.65:41687/ using document root <none> and password file <none>
I20260806 04:58:14.128110 21270 fs_manager.cc:359] Metadata directory not provided
I20260806 04:58:14.128250 21270 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 04:58:14.128598 21270 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 04:58:14.131524 21270 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785992227050689-9601-0/minicluster-data/ts-0/data/instance:
uuid: "8f45f7b26a8f43799df89071dca99b7a"
format_stamp: "Formatted at 2026-08-06 04:58:14 on dist-test-slave-x4qh"
server_key: "e19de763bff9ad8b3a6f418a12842eab"
server_key_iv: "52b62d1e8414f3b6053fbbe11a45384a"
server_key_version: "encryptionkey@0"
I20260806 04:58:14.132448 21270 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785992227050689-9601-0/minicluster-data/ts-0/wal/instance:
uuid: "8f45f7b26a8f43799df89071dca99b7a"
format_stamp: "Formatted at 2026-08-06 04:58:14 on dist-test-slave-x4qh"
server_key: "e19de763bff9ad8b3a6f418a12842eab"
server_key_iv: "52b62d1e8414f3b6053fbbe11a45384a"
server_key_version: "encryptionkey@0"
I20260806 04:58:14.137750 21270 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260806 04:58:14.141243 21292 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 04:58:14.142586 21270 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260806 04:58:14.142725 21270 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785992227050689-9601-0/minicluster-data/ts-0/data,/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785992227050689-9601-0/minicluster-data/ts-0/wal
uuid: "8f45f7b26a8f43799df89071dca99b7a"
format_stamp: "Formatted at 2026-08-06 04:58:14 on dist-test-slave-x4qh"
server_key: "e19de763bff9ad8b3a6f418a12842eab"
server_key_iv: "52b62d1e8414f3b6053fbbe11a45384a"
server_key_version: "encryptionkey@0"
I20260806 04:58:14.142916 21270 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785992227050689-9601-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785992227050689-9601-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785992227050689-9601-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 04:58:14.180091 21270 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 04:58:14.184680 21270 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 04:58:14.185014 21270 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 04:58:14.187687 21270 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 04:58:14.187774 21270 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 04:58:14.187954 21270 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 04:58:14.188015 21270 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 04:58:14 dist-test-slave-x4qh krb5kdc[21106](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992294, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.96.65@KRBTEST.COM for oryx/127.9.96.126@KRBTEST.COM
I20260806 04:58:14.212263 21306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:58:14.188468 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.96.65:58521 (local address 127.9.96.126:40621)
0806 04:58:14.190665 (+ 2197us) server_negotiation.cc:206] Beginning negotiation
0806 04:58:14.190672 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 04:58:14.190695 (+ 23us) server_negotiation.cc:407] Connection header received
0806 04:58:14.192482 (+ 1787us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:58:14.192488 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:58:14.192589 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:58:14.192714 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:58:14.194360 (+ 1646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:14.195478 (+ 1118us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:14.198807 (+ 3329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:14.199363 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:14.199537 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:58:14.204583 (+ 5046us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:58:14.204590 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:58:14.204595 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:58:14.204636 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:58:14.208479 (+ 3843us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:58:14.209391 (+ 912us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:58:14.209401 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:58:14.209406 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:58:14.209616 (+ 210us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:58:14.210158 (+ 542us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:58:14.210165 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:58:14.210169 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:58:14.210554 (+ 385us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:58:14.210694 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0806 04:58:14.211652 (+ 958us) server_negotiation.cc:299] Negotiation successful
0806 04:58:14.211882 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2067,"thread_start_us":1930,"threads_started":1}
I20260806 04:58:14.213197 21308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:58:14.188973 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:40621 (local address 127.9.96.65:58521)
0806 04:58:14.189874 (+ 901us) negotiation.cc:107] Waiting for socket to connect
0806 04:58:14.189919 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0806 04:58:14.192176 (+ 2257us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:58:14.192976 (+ 800us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:58:14.192983 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:58:14.193618 (+ 635us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:58:14.194180 (+ 562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:58:14.194197 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:58:14.195678 (+ 1481us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:58:14.195687 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:58:14.198607 (+ 2920us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:58:14.198633 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:58:14.199661 (+ 1028us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:58:14.199672 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:58:14.200046 (+ 374us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:58:14.200813 (+ 767us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:58:14.200858 (+ 45us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:58:14.204344 (+ 3486us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 04:58:14.208652 (+ 4308us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:58:14.208661 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:58:14.208680 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:58:14.209207 (+ 527us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:58:14.209788 (+ 581us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:58:14.209793 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:58:14.209799 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:58:14.210022 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:58:14.210684 (+ 662us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 04:58:14.210692 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 04:58:14.211142 (+ 450us) client_negotiation.cc:769] Sending connection context
0806 04:58:14.211567 (+ 425us) client_negotiation.cc:240] Negotiation successful
0806 04:58:14.212071 (+ 504us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":645,"thread_start_us":437,"threads_started":1}
I20260806 04:58:14.237084 21306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:58:14.220968 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.96.65:52477 (local address 127.9.96.126:40621)
0806 04:58:14.221144 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0806 04:58:14.221150 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 04:58:14.222336 (+ 1186us) server_negotiation.cc:407] Connection header received
0806 04:58:14.222619 (+ 283us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:58:14.222625 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:58:14.222719 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:58:14.222808 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:58:14.224771 (+ 1963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:14.225645 (+ 874us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:14.227770 (+ 2125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:14.228341 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:14.228494 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:58:14.230794 (+ 2300us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:58:14.230799 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:58:14.230802 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:58:14.230830 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:58:14.233956 (+ 3126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:58:14.234898 (+ 942us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:58:14.234906 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:58:14.234911 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:58:14.235086 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:58:14.235670 (+ 584us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:58:14.235675 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:58:14.235678 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:58:14.235960 (+ 282us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:58:14.236078 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0806 04:58:14.236654 (+ 576us) server_negotiation.cc:299] Negotiation successful
0806 04:58:14.236807 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260806 04:58:14.237711 21395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:58:14.221096 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:40621 (local address 127.9.96.65:52477)
0806 04:58:14.222210 (+ 1114us) negotiation.cc:107] Waiting for socket to connect
0806 04:58:14.222253 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0806 04:58:14.222419 (+ 166us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:58:14.223039 (+ 620us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:58:14.223047 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:58:14.223908 (+ 861us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:58:14.224554 (+ 646us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:58:14.224573 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:58:14.225783 (+ 1210us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:58:14.225795 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:58:14.227605 (+ 1810us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:58:14.227617 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:58:14.228536 (+ 919us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:58:14.228542 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:58:14.228703 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:58:14.229352 (+ 649us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:58:14.229367 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:58:14.230573 (+ 1206us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 04:58:14.234119 (+ 3546us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:58:14.234126 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:58:14.234129 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:58:14.234743 (+ 614us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:58:14.235232 (+ 489us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:58:14.235248 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:58:14.235254 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:58:14.235522 (+ 268us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:58:14.236095 (+ 573us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 04:58:14.236103 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 04:58:14.236374 (+ 271us) client_negotiation.cc:769] Sending connection context
0806 04:58:14.236556 (+ 182us) client_negotiation.cc:240] Negotiation successful
0806 04:58:14.237030 (+ 474us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":814,"thread_start_us":606,"threads_started":1}
I20260806 04:58:14.244948 21270 rpc_server.cc:307] RPC server started. Bound to: 127.9.96.65:39027
I20260806 04:58:14.244978 21474 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.96.65:39027 every 8 connection(s)
I20260806 04:58:14.246269 21270 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785992227050689-9601-0/minicluster-data/ts-0/data/info.pb
I20260806 04:58:14.248332 21476 heartbeater.cc:347] Connected to a master server at 127.9.96.126:40621
I20260806 04:58:14.248500 21476 heartbeater.cc:464] Registering TS with master...
I20260806 04:58:14.249012 21476 heartbeater.cc:511] Master 127.9.96.126:40621 requested a full tablet report, sending...
I20260806 04:58:14.250689 21171 ts_manager.cc:194] Registered new tserver with Master: 8f45f7b26a8f43799df89071dca99b7a (127.9.96.65:39027)
I20260806 04:58:14.252122 21171 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.9.96.65@KRBTEST.COM'} at 127.9.96.65:58521
I20260806 04:58:14.253145 9601 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu as pid 21270
I20260806 04:58:14.253372 9601 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785992227050689-9601-0/minicluster-data/ts-0/wal/instance
I20260806 04:58:14.253975 9601 external_mini_cluster.cc:1428] Setting key cbb7cd4995d387a110456ba038ae0481
WARNING: no policy specified for oryx/127.9.96.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.9.96.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.9.96.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785992227050689-9601-0/krb5kdc/oryx_127.9.96.66.keytab.
Entry for principal oryx/127.9.96.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785992227050689-9601-0/krb5kdc/oryx_127.9.96.66.keytab.
I20260806 04:58:14.304816 9601 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
/tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785992227050689-9601-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785992227050689-9601-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785992227050689-9601-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785992227050689-9601-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.96.66:0
--local_ip_for_outbound_sockets=127.9.96.66
--webserver_interface=127.9.96.66
--webserver_port=0
--tserver_master_addrs=127.9.96.126:40621
--keytab_file=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785992227050689-9601-0/krb5kdc/oryx_127.9.96.66.keytab
--principal=oryx/127.9.96.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.96.84:46011
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785992227050689-9601-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785992227050689-9601-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785992227050689-9601-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 04:58:14.629396 21482 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 04:58:14.629712 21482 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 04:58:14.629889 21482 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 04:58:14.636981 21482 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 04:58:14.637123 21482 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 04:58:14.637264 21482 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 04:58:14.637403 21482 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.96.66
I20260806 04:58:14.647181 21482 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.96.84:46011
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785992227050689-9601-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785992227050689-9601-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785992227050689-9601-0/krb5kdc/oryx_127.9.96.66.keytab
--principal=oryx/127.9.96.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.96.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785992227050689-9601-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.96.66
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.9.96.126:40621
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.96.66
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785992227050689-9601-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 03:56:12 UTC on ccycloud.quasar-guijbm.root.comops.site
build id 4569618
ASAN enabled
I20260806 04:58:14.648451 21482 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 04:58:14.649689 21482 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 04:58:14.665499 21495 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:58:14.665791 21494 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:58:14.666531 21498 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:58:14.667043 21482 server_base.cc:1034] running on GCE node
I20260806 04:58:14.667659 21482 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 04:58:14.668565 21482 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 04:58:14.669831 21482 hybrid_clock.cc:648] HybridClock initialized: now 1785992294669746 us; error 51 us; skew 500 ppm
I20260806 04:58:14.670127 21482 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 04:58:14 dist-test-slave-x4qh krb5kdc[21106](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992294, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.96.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 04:58:14.673645 21482 init.cc:377] Logged in from keytab as oryx/127.9.96.66@KRBTEST.COM (short username oryx)
I20260806 04:58:14.675289 21482 webserver.cc:460] Webserver started at http://127.9.96.66:35429/ using document root <none> and password file <none>
I20260806 04:58:14.675983 21482 fs_manager.cc:359] Metadata directory not provided
I20260806 04:58:14.676122 21482 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 04:58:14.676456 21482 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 04:58:14.679215 21482 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785992227050689-9601-0/minicluster-data/ts-1/data/instance:
uuid: "0c634c0504d446008ab407bb82882031"
format_stamp: "Formatted at 2026-08-06 04:58:14 on dist-test-slave-x4qh"
server_key: "df19de4f3687e1826c95e6c7a90a8122"
server_key_iv: "228ae44c8dd32072d690edef9471ddad"
server_key_version: "encryptionkey@0"
I20260806 04:58:14.680091 21482 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785992227050689-9601-0/minicluster-data/ts-1/wal/instance:
uuid: "0c634c0504d446008ab407bb82882031"
format_stamp: "Formatted at 2026-08-06 04:58:14 on dist-test-slave-x4qh"
server_key: "df19de4f3687e1826c95e6c7a90a8122"
server_key_iv: "228ae44c8dd32072d690edef9471ddad"
server_key_version: "encryptionkey@0"
I20260806 04:58:14.685534 21482 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260806 04:58:14.690636 21508 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 04:58:14.692299 21482 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260806 04:58:14.692482 21482 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785992227050689-9601-0/minicluster-data/ts-1/data,/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785992227050689-9601-0/minicluster-data/ts-1/wal
uuid: "0c634c0504d446008ab407bb82882031"
format_stamp: "Formatted at 2026-08-06 04:58:14 on dist-test-slave-x4qh"
server_key: "df19de4f3687e1826c95e6c7a90a8122"
server_key_iv: "228ae44c8dd32072d690edef9471ddad"
server_key_version: "encryptionkey@0"
I20260806 04:58:14.692785 21482 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785992227050689-9601-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785992227050689-9601-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785992227050689-9601-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 04:58:14.738379 21482 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 04:58:14.745018 21482 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 04:58:14.745525 21482 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 04:58:14.750054 21482 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 04:58:14.750164 21482 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 04:58:14.750361 21482 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 04:58:14.750445 21482 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.001s sys 0.000s
Aug 06 04:58:14 dist-test-slave-x4qh krb5kdc[21106](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992294, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.96.66@KRBTEST.COM for oryx/127.9.96.126@KRBTEST.COM
I20260806 04:58:14.781759 21519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:58:14.749773 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.96.66:41029 (local address 127.9.96.126:40621)
0806 04:58:14.765210 (+ 15437us) server_negotiation.cc:206] Beginning negotiation
0806 04:58:14.765218 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 04:58:14.765238 (+ 20us) server_negotiation.cc:407] Connection header received
0806 04:58:14.765310 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:58:14.765314 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:58:14.765388 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:58:14.765524 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:58:14.767243 (+ 1719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:14.768154 (+ 911us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:14.770042 (+ 1888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:14.770747 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:14.770892 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:58:14.775075 (+ 4183us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:58:14.775079 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:58:14.775083 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:58:14.775112 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:58:14.778612 (+ 3500us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:58:14.779397 (+ 785us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:58:14.779405 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:58:14.779409 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:58:14.779540 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:58:14.780005 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:58:14.780011 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:58:14.780014 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:58:14.780283 (+ 269us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:58:14.780492 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0806 04:58:14.781155 (+ 663us) server_negotiation.cc:299] Negotiation successful
0806 04:58:14.781380 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15307,"thread_start_us":15855,"threads_started":1}
I20260806 04:58:14.782734 21522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:58:14.750384 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:40621 (local address 127.9.96.66:41029)
0806 04:58:14.760117 (+ 9733us) negotiation.cc:107] Waiting for socket to connect
0806 04:58:14.760174 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0806 04:58:14.764530 (+ 4356us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:58:14.765783 (+ 1253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:58:14.765794 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:58:14.766317 (+ 523us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:58:14.767056 (+ 739us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:58:14.767075 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:58:14.768329 (+ 1254us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:58:14.768334 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:58:14.769896 (+ 1562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:58:14.769908 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:58:14.771288 (+ 1380us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:58:14.771311 (+ 23us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:58:14.771635 (+ 324us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:58:14.772305 (+ 670us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:58:14.772326 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:58:14.774851 (+ 2525us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 04:58:14.778771 (+ 3920us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:58:14.778780 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:58:14.778795 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:58:14.779236 (+ 441us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:58:14.779670 (+ 434us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:58:14.779675 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:58:14.779678 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:58:14.779872 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:58:14.780450 (+ 578us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 04:58:14.780458 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 04:58:14.780778 (+ 320us) client_negotiation.cc:769] Sending connection context
0806 04:58:14.781061 (+ 283us) client_negotiation.cc:240] Negotiation successful
0806 04:58:14.781532 (+ 471us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9368,"thread_start_us":12851,"threads_started":1}
I20260806 04:58:14.804585 21519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:58:14.788193 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.96.66:59189 (local address 127.9.96.126:40621)
0806 04:58:14.788358 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0806 04:58:14.788362 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 04:58:14.792670 (+ 4308us) server_negotiation.cc:407] Connection header received
0806 04:58:14.792879 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:58:14.792885 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:58:14.792951 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:58:14.793053 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:58:14.793947 (+ 894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:14.794786 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:14.796384 (+ 1598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:14.796802 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:14.796949 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:58:14.798928 (+ 1979us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:58:14.798932 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:58:14.798935 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:58:14.798964 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:58:14.801813 (+ 2849us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:58:14.802470 (+ 657us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:58:14.802475 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:58:14.802478 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:58:14.802601 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:58:14.802981 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:58:14.802986 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:58:14.802989 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:58:14.803322 (+ 333us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:58:14.803538 (+ 216us) server_negotiation.cc:1035] Waiting for connection context
0806 04:58:14.803987 (+ 449us) server_negotiation.cc:299] Negotiation successful
0806 04:58:14.804214 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260806 04:58:14.804665 21547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:58:14.791821 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:40621 (local address 127.9.96.66:59189)
0806 04:58:14.792561 (+ 740us) negotiation.cc:107] Waiting for socket to connect
0806 04:58:14.792582 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0806 04:58:14.792741 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:58:14.793080 (+ 339us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:58:14.793085 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:58:14.793524 (+ 439us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:58:14.793798 (+ 274us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:58:14.793808 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:58:14.794987 (+ 1179us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:58:14.794990 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:58:14.796235 (+ 1245us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:58:14.796247 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:58:14.796991 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:58:14.796997 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:58:14.797155 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:58:14.797830 (+ 675us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:58:14.797847 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:58:14.798729 (+ 882us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 04:58:14.801956 (+ 3227us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:58:14.801962 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:58:14.801966 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:58:14.802340 (+ 374us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:58:14.802719 (+ 379us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:58:14.802724 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:58:14.802728 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:58:14.802857 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:58:14.803517 (+ 660us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 04:58:14.803522 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 04:58:14.803697 (+ 175us) client_negotiation.cc:769] Sending connection context
0806 04:58:14.803872 (+ 175us) client_negotiation.cc:240] Negotiation successful
0806 04:58:14.804183 (+ 311us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":588,"thread_start_us":431,"threads_started":1}
I20260806 04:58:14.842700 21482 rpc_server.cc:307] RPC server started. Bound to: 127.9.96.66:46591
I20260806 04:58:14.842712 21693 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.96.66:46591 every 8 connection(s)
I20260806 04:58:14.844024 21482 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785992227050689-9601-0/minicluster-data/ts-1/data/info.pb
I20260806 04:58:14.846428 21695 heartbeater.cc:347] Connected to a master server at 127.9.96.126:40621
I20260806 04:58:14.846660 21695 heartbeater.cc:464] Registering TS with master...
I20260806 04:58:14.847426 21695 heartbeater.cc:511] Master 127.9.96.126:40621 requested a full tablet report, sending...
I20260806 04:58:14.848667 9601 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu as pid 21482
I20260806 04:58:14.848871 9601 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785992227050689-9601-0/minicluster-data/ts-1/wal/instance
I20260806 04:58:14.848866 21171 ts_manager.cc:194] Registered new tserver with Master: 0c634c0504d446008ab407bb82882031 (127.9.96.66:46591)
I20260806 04:58:14.849300 9601 external_mini_cluster.cc:1428] Setting key f533f4651cadcba846bfcced8320ab08
I20260806 04:58:14.849875 21171 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.9.96.66@KRBTEST.COM'} at 127.9.96.66:41029
WARNING: no policy specified for oryx/127.9.96.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.9.96.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.9.96.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785992227050689-9601-0/krb5kdc/oryx_127.9.96.67.keytab.
Entry for principal oryx/127.9.96.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785992227050689-9601-0/krb5kdc/oryx_127.9.96.67.keytab.
I20260806 04:58:14.902956 9601 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
/tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785992227050689-9601-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785992227050689-9601-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785992227050689-9601-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785992227050689-9601-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.96.67:0
--local_ip_for_outbound_sockets=127.9.96.67
--webserver_interface=127.9.96.67
--webserver_port=0
--tserver_master_addrs=127.9.96.126:40621
--keytab_file=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785992227050689-9601-0/krb5kdc/oryx_127.9.96.67.keytab
--principal=oryx/127.9.96.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.96.84:46011
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785992227050689-9601-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785992227050689-9601-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785992227050689-9601-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 04:58:15.228931 21701 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 04:58:15.229183 21701 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 04:58:15.229277 21701 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 04:58:15.236129 21701 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 04:58:15.236227 21701 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 04:58:15.236307 21701 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 04:58:15.236387 21701 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.96.67
I20260806 04:58:15.245536 21701 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.96.84:46011
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785992227050689-9601-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785992227050689-9601-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785992227050689-9601-0/krb5kdc/oryx_127.9.96.67.keytab
--principal=oryx/127.9.96.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.96.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785992227050689-9601-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.96.67
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.9.96.126:40621
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.96.67
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785992227050689-9601-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 03:56:12 UTC on ccycloud.quasar-guijbm.root.comops.site
build id 4569618
ASAN enabled
I20260806 04:58:15.246869 21701 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 04:58:15.248364 21701 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 04:58:15.254820 21476 heartbeater.cc:503] Master 127.9.96.126:40621 was elected leader, sending a full tablet report...
W20260806 04:58:15.262065 21706 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:58:15.262029 21707 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:58:15.262152 21712 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:58:15.262732 21701 server_base.cc:1034] running on GCE node
I20260806 04:58:15.263307 21701 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 04:58:15.264137 21701 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 04:58:15.265357 21701 hybrid_clock.cc:648] HybridClock initialized: now 1785992295265291 us; error 39 us; skew 500 ppm
I20260806 04:58:15.265606 21701 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 04:58:15 dist-test-slave-x4qh krb5kdc[21106](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992295, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.96.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 04:58:15.268819 21701 init.cc:377] Logged in from keytab as oryx/127.9.96.67@KRBTEST.COM (short username oryx)
I20260806 04:58:15.270114 21701 webserver.cc:460] Webserver started at http://127.9.96.67:33059/ using document root <none> and password file <none>
I20260806 04:58:15.270771 21701 fs_manager.cc:359] Metadata directory not provided
I20260806 04:58:15.270896 21701 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 04:58:15.271267 21701 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 04:58:15.274113 21701 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785992227050689-9601-0/minicluster-data/ts-2/data/instance:
uuid: "ebc08b93691f474698e93a5cc45baa6c"
format_stamp: "Formatted at 2026-08-06 04:58:15 on dist-test-slave-x4qh"
server_key: "6d2fe7c0dfcba2efc34d64535c2667fe"
server_key_iv: "5cfc4a46150dbf45156d866050c75535"
server_key_version: "encryptionkey@0"
I20260806 04:58:15.274883 21701 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785992227050689-9601-0/minicluster-data/ts-2/wal/instance:
uuid: "ebc08b93691f474698e93a5cc45baa6c"
format_stamp: "Formatted at 2026-08-06 04:58:15 on dist-test-slave-x4qh"
server_key: "6d2fe7c0dfcba2efc34d64535c2667fe"
server_key_iv: "5cfc4a46150dbf45156d866050c75535"
server_key_version: "encryptionkey@0"
I20260806 04:58:15.280009 21701 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260806 04:58:15.283509 21723 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 04:58:15.284960 21701 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260806 04:58:15.285090 21701 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785992227050689-9601-0/minicluster-data/ts-2/data,/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785992227050689-9601-0/minicluster-data/ts-2/wal
uuid: "ebc08b93691f474698e93a5cc45baa6c"
format_stamp: "Formatted at 2026-08-06 04:58:15 on dist-test-slave-x4qh"
server_key: "6d2fe7c0dfcba2efc34d64535c2667fe"
server_key_iv: "5cfc4a46150dbf45156d866050c75535"
server_key_version: "encryptionkey@0"
I20260806 04:58:15.285322 21701 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785992227050689-9601-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785992227050689-9601-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785992227050689-9601-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 04:58:15.298935 21701 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 04:58:15.302675 21701 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 04:58:15.302893 21701 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 04:58:15.305922 21701 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 04:58:15.306034 21701 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 04:58:15.306181 21701 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 04:58:15.306252 21701 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 04:58:15 dist-test-slave-x4qh krb5kdc[21106](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992295, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.96.67@KRBTEST.COM for oryx/127.9.96.126@KRBTEST.COM
I20260806 04:58:15.330039 21737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:58:15.306319 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.96.67:49775 (local address 127.9.96.126:40621)
0806 04:58:15.306807 (+ 488us) server_negotiation.cc:206] Beginning negotiation
0806 04:58:15.306816 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 04:58:15.308025 (+ 1209us) server_negotiation.cc:407] Connection header received
0806 04:58:15.310052 (+ 2027us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:58:15.310058 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:58:15.310164 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:58:15.310293 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:58:15.312539 (+ 2246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:15.313922 (+ 1383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:15.317174 (+ 3252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:15.317587 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:15.317740 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:58:15.322341 (+ 4601us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:58:15.322349 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:58:15.322355 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:58:15.322416 (+ 61us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:58:15.325882 (+ 3466us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:58:15.326963 (+ 1081us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:58:15.326971 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:58:15.326977 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:58:15.327191 (+ 214us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:58:15.327856 (+ 665us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:58:15.327861 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:58:15.327864 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:58:15.328259 (+ 395us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:58:15.328418 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0806 04:58:15.329403 (+ 985us) server_negotiation.cc:299] Negotiation successful
0806 04:58:15.329628 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":344,"thread_start_us":244,"threads_started":1}
I20260806 04:58:15.331027 21740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:58:15.306942 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:40621 (local address 127.9.96.67:49775)
0806 04:58:15.307853 (+ 911us) negotiation.cc:107] Waiting for socket to connect
0806 04:58:15.307899 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0806 04:58:15.309764 (+ 1865us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:58:15.310543 (+ 779us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:58:15.310550 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:58:15.311287 (+ 737us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:58:15.312305 (+ 1018us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:58:15.312324 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:58:15.314140 (+ 1816us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:58:15.314167 (+ 27us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:58:15.317026 (+ 2859us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:58:15.317042 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:58:15.317845 (+ 803us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:58:15.317852 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:58:15.318164 (+ 312us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:58:15.319146 (+ 982us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:58:15.319181 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:58:15.322095 (+ 2914us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 04:58:15.326100 (+ 4005us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:58:15.326112 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:58:15.326132 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:58:15.326779 (+ 647us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:58:15.327355 (+ 576us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:58:15.327361 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:58:15.327364 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:58:15.327708 (+ 344us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:58:15.328389 (+ 681us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 04:58:15.328397 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 04:58:15.328921 (+ 524us) client_negotiation.cc:769] Sending connection context
0806 04:58:15.329287 (+ 366us) client_negotiation.cc:240] Negotiation successful
0806 04:58:15.329797 (+ 510us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":678,"thread_start_us":484,"threads_started":1}
I20260806 04:58:15.353340 21841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:58:15.337503 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:40621 (local address 127.9.96.67:43927)
0806 04:58:15.338289 (+ 786us) negotiation.cc:107] Waiting for socket to connect
0806 04:58:15.338312 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0806 04:58:15.338478 (+ 166us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:58:15.338814 (+ 336us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:58:15.338821 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:58:15.339596 (+ 775us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:58:15.340153 (+ 557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:58:15.340171 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:58:15.341631 (+ 1460us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:58:15.341639 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:58:15.343796 (+ 2157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:58:15.343835 (+ 39us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:58:15.344901 (+ 1066us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:58:15.344911 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:58:15.345171 (+ 260us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:58:15.345920 (+ 749us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:58:15.345961 (+ 41us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:58:15.346731 (+ 770us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 04:58:15.350692 (+ 3961us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:58:15.350697 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:58:15.350701 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:58:15.351202 (+ 501us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:58:15.351645 (+ 443us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:58:15.351649 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:58:15.351653 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:58:15.351865 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:58:15.352521 (+ 656us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 04:58:15.352524 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 04:58:15.352677 (+ 153us) client_negotiation.cc:769] Sending connection context
0806 04:58:15.352791 (+ 114us) client_negotiation.cc:240] Negotiation successful
0806 04:58:15.353024 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":602,"thread_start_us":424,"threads_started":1}
I20260806 04:58:15.354050 21737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:58:15.337562 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.96.67:43927 (local address 127.9.96.126:40621)
0806 04:58:15.337795 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0806 04:58:15.337804 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 04:58:15.338417 (+ 613us) server_negotiation.cc:407] Connection header received
0806 04:58:15.338594 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:58:15.338597 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:58:15.338654 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:58:15.338730 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:58:15.340375 (+ 1645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:15.341252 (+ 877us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:15.343983 (+ 2731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:15.344675 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:15.344771 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:58:15.346988 (+ 2217us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:58:15.346995 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:58:15.346999 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:58:15.347035 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:58:15.350553 (+ 3518us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:58:15.351329 (+ 776us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:58:15.351334 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:58:15.351338 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:58:15.351522 (+ 184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:58:15.352011 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:58:15.352019 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:58:15.352024 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:58:15.352394 (+ 370us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:58:15.352549 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0806 04:58:15.352941 (+ 392us) server_negotiation.cc:299] Negotiation successful
0806 04:58:15.353176 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260806 04:58:15.361374 21701 rpc_server.cc:307] RPC server started. Bound to: 127.9.96.67:36799
I20260806 04:58:15.361393 21901 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.96.67:36799 every 8 connection(s)
I20260806 04:58:15.362505 21701 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785992227050689-9601-0/minicluster-data/ts-2/data/info.pb
I20260806 04:58:15.364559 21902 heartbeater.cc:347] Connected to a master server at 127.9.96.126:40621
I20260806 04:58:15.364789 21902 heartbeater.cc:464] Registering TS with master...
I20260806 04:58:15.364796 9601 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu as pid 21701
I20260806 04:58:15.364988 9601 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785992227050689-9601-0/minicluster-data/ts-2/wal/instance
I20260806 04:58:15.365454 9601 external_mini_cluster.cc:1428] Setting key 4705cdeaf5e188c5e9674e79760c4dd4
I20260806 04:58:15.365484 21902 heartbeater.cc:511] Master 127.9.96.126:40621 requested a full tablet report, sending...
I20260806 04:58:15.367103 21171 ts_manager.cc:194] Registered new tserver with Master: ebc08b93691f474698e93a5cc45baa6c (127.9.96.67:36799)
I20260806 04:58:15.368037 21171 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.9.96.67@KRBTEST.COM'} at 127.9.96.67:49775
I20260806 04:58:15.368688 9601 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 06 04:58:15 dist-test-slave-x4qh krb5kdc[21106](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992295, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 06 04:58:15 dist-test-slave-x4qh krb5kdc[21106](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.9.96.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 04:58:15 dist-test-slave-x4qh krb5kdc[21106](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.9.96.126@KRBTEST.COM, Server not found in Kerberos database
W20260806 04:58:15.424640 21737 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 04:58:15.413507 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57206 (local address 127.9.96.126:40621)
0806 04:58:15.413745 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0806 04:58:15.413753 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 04:58:15.413944 (+ 191us) server_negotiation.cc:407] Connection header received
0806 04:58:15.414177 (+ 233us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:58:15.414183 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:58:15.414257 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:58:15.414386 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:58:15.415903 (+ 1517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:15.416778 (+ 875us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:15.417928 (+ 1150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:15.418347 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:15.418507 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:58:15.424233 (+ 5726us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57206: BlockingRecv error: recv got EOF from 127.0.0.1:57206 (error 108)
Metrics: {"server-negotiator.queue_time_us":85}
Aug 06 04:58:15 dist-test-slave-x4qh krb5kdc[21106](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992295, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.9.96.126@KRBTEST.COM
I20260806 04:58:15.440675 21737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:58:15.427256 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57216 (local address 127.9.96.126:40621)
0806 04:58:15.427488 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0806 04:58:15.427493 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 04:58:15.427688 (+ 195us) server_negotiation.cc:407] Connection header received
0806 04:58:15.427804 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:58:15.427809 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:58:15.427883 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:58:15.427962 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:58:15.428973 (+ 1011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:15.429680 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:15.431409 (+ 1729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:15.431834 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:15.431978 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:58:15.435698 (+ 3720us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:58:15.435703 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:58:15.435706 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:58:15.435735 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:58:15.438037 (+ 2302us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:58:15.438721 (+ 684us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:58:15.438727 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:58:15.438731 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:58:15.438862 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:58:15.439243 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:58:15.439248 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:58:15.439251 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:58:15.439574 (+ 323us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:58:15.439867 (+ 293us) server_negotiation.cc:1035] Waiting for connection context
0806 04:58:15.440153 (+ 286us) server_negotiation.cc:299] Negotiation successful
0806 04:58:15.440380 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260806 04:58:15.444597 21171 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:57216:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 04:58:15.451858 21171 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 04:58:15.490633 21932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:58:15.463402 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.66:46591 (local address 127.0.0.1:34038)
0806 04:58:15.481744 (+ 18342us) negotiation.cc:107] Waiting for socket to connect
0806 04:58:15.481765 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0806 04:58:15.482011 (+ 246us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:58:15.482927 (+ 916us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:58:15.482934 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:58:15.482979 (+ 45us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 04:58:15.483550 (+ 571us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:58:15.483568 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:58:15.485494 (+ 1926us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:58:15.485501 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:58:15.487932 (+ 2431us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:58:15.487945 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:58:15.489453 (+ 1508us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:58:15.489460 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:58:15.489644 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:58:15.489692 (+ 48us) client_negotiation.cc:769] Sending connection context
0806 04:58:15.489830 (+ 138us) client_negotiation.cc:240] Negotiation successful
0806 04:58:15.490010 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":18172,"mutex_wait_us":235,"spinlock_wait_cycles":1920,"thread_start_us":18243,"threads_started":1}
I20260806 04:58:15.490645 21931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:58:15.463771 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34038 (local address 127.9.96.66:46591)
0806 04:58:15.481362 (+ 17591us) server_negotiation.cc:206] Beginning negotiation
0806 04:58:15.481373 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 04:58:15.481881 (+ 508us) server_negotiation.cc:407] Connection header received
0806 04:58:15.482268 (+ 387us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:58:15.482275 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:58:15.482523 (+ 248us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:58:15.482696 (+ 173us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 04:58:15.483805 (+ 1109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:15.485323 (+ 1518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:15.488093 (+ 2770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:15.489263 (+ 1170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:15.489384 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:58:15.489580 (+ 196us) server_negotiation.cc:1035] Waiting for connection context
0806 04:58:15.490183 (+ 603us) server_negotiation.cc:299] Negotiation successful
0806 04:58:15.490362 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17396,"thread_start_us":17247,"threads_started":1}
I20260806 04:58:15.491218 21928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:58:15.463279 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.65:39027 (local address 127.0.0.1:35808)
0806 04:58:15.478105 (+ 14826us) negotiation.cc:107] Waiting for socket to connect
0806 04:58:15.478164 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0806 04:58:15.478388 (+ 224us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:58:15.482914 (+ 4526us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:58:15.482920 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:58:15.482949 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 04:58:15.483486 (+ 537us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:58:15.483504 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:58:15.485528 (+ 2024us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:58:15.485532 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:58:15.488525 (+ 2993us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:58:15.488540 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:58:15.490397 (+ 1857us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:58:15.490403 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:58:15.490565 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:58:15.490609 (+ 44us) client_negotiation.cc:769] Sending connection context
0806 04:58:15.490684 (+ 75us) client_negotiation.cc:240] Negotiation successful
0806 04:58:15.490813 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14507,"mutex_wait_us":178,"spinlock_wait_cycles":8448,"thread_start_us":15292,"threads_started":1}
I20260806 04:58:15.492177 21930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:58:15.463467 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35808 (local address 127.9.96.65:39027)
0806 04:58:15.482201 (+ 18734us) server_negotiation.cc:206] Beginning negotiation
0806 04:58:15.482208 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 04:58:15.482264 (+ 56us) server_negotiation.cc:407] Connection header received
0806 04:58:15.482369 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:58:15.482379 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:58:15.482616 (+ 237us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:58:15.483520 (+ 904us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 04:58:15.483726 (+ 206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:15.485317 (+ 1591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:15.488744 (+ 3427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:15.490159 (+ 1415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:15.491465 (+ 1306us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:58:15.491651 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0806 04:58:15.491720 (+ 69us) server_negotiation.cc:299] Negotiation successful
0806 04:58:15.491880 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18571,"thread_start_us":18427,"threads_started":1}
I20260806 04:58:15.492342 21933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:58:15.465836 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33152 (local address 127.9.96.67:36799)
0806 04:58:15.480843 (+ 15007us) server_negotiation.cc:206] Beginning negotiation
0806 04:58:15.480855 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 04:58:15.482072 (+ 1217us) server_negotiation.cc:407] Connection header received
0806 04:58:15.482501 (+ 429us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:58:15.482508 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:58:15.482690 (+ 182us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:58:15.482820 (+ 130us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 04:58:15.483888 (+ 1068us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:15.485251 (+ 1363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:15.488704 (+ 3453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:15.490997 (+ 2293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:15.491123 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:58:15.491348 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0806 04:58:15.491852 (+ 504us) server_negotiation.cc:299] Negotiation successful
0806 04:58:15.492008 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14778,"thread_start_us":14557,"threads_started":1}
I20260806 04:58:15.492218 21929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:58:15.465482 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.67:36799 (local address 127.0.0.1:33152)
0806 04:58:15.481956 (+ 16474us) negotiation.cc:107] Waiting for socket to connect
0806 04:58:15.481973 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 04:58:15.482326 (+ 353us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:58:15.483150 (+ 824us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:58:15.483156 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:58:15.483178 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 04:58:15.483705 (+ 527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:58:15.483717 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:58:15.485428 (+ 1711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:58:15.485433 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:58:15.488525 (+ 3092us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:58:15.488540 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:58:15.491219 (+ 2679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:58:15.491225 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:58:15.491615 (+ 390us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:58:15.491647 (+ 32us) client_negotiation.cc:769] Sending connection context
0806 04:58:15.491739 (+ 92us) client_negotiation.cc:240] Negotiation successful
0806 04:58:15.491858 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":16327,"mutex_wait_us":210,"thread_start_us":15937,"threads_started":1}
I20260806 04:58:15.494705 21586 tablet_service.cc:1467] Processing CreateTablet for tablet 18eaa3f05cb44595b8d9702230c30ab4 (DEFAULT_TABLE table=test-table [id=abfe56ad98794e8398d938440634754d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 04:58:15.495527 21369 tablet_service.cc:1467] Processing CreateTablet for tablet 18eaa3f05cb44595b8d9702230c30ab4 (DEFAULT_TABLE table=test-table [id=abfe56ad98794e8398d938440634754d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 04:58:15.495658 21802 tablet_service.cc:1467] Processing CreateTablet for tablet 18eaa3f05cb44595b8d9702230c30ab4 (DEFAULT_TABLE table=test-table [id=abfe56ad98794e8398d938440634754d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 04:58:15.496188 21586 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 18eaa3f05cb44595b8d9702230c30ab4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 04:58:15.496824 21802 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 18eaa3f05cb44595b8d9702230c30ab4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 04:58:15.496841 21369 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 18eaa3f05cb44595b8d9702230c30ab4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 04:58:15.515625 21938 tablet_bootstrap.cc:492] T 18eaa3f05cb44595b8d9702230c30ab4 P 8f45f7b26a8f43799df89071dca99b7a: Bootstrap starting.
I20260806 04:58:15.516508 21935 tablet_bootstrap.cc:492] T 18eaa3f05cb44595b8d9702230c30ab4 P 0c634c0504d446008ab407bb82882031: Bootstrap starting.
I20260806 04:58:15.518175 21934 tablet_bootstrap.cc:492] T 18eaa3f05cb44595b8d9702230c30ab4 P ebc08b93691f474698e93a5cc45baa6c: Bootstrap starting.
I20260806 04:58:15.520088 21938 tablet_bootstrap.cc:654] T 18eaa3f05cb44595b8d9702230c30ab4 P 8f45f7b26a8f43799df89071dca99b7a: Neither blocks nor log segments found. Creating new log.
I20260806 04:58:15.521222 21935 tablet_bootstrap.cc:654] T 18eaa3f05cb44595b8d9702230c30ab4 P 0c634c0504d446008ab407bb82882031: Neither blocks nor log segments found. Creating new log.
I20260806 04:58:15.521517 21938 log.cc:824] T 18eaa3f05cb44595b8d9702230c30ab4 P 8f45f7b26a8f43799df89071dca99b7a: Log is configured to *not* fsync() on all Append() calls
I20260806 04:58:15.521559 21934 tablet_bootstrap.cc:654] T 18eaa3f05cb44595b8d9702230c30ab4 P ebc08b93691f474698e93a5cc45baa6c: Neither blocks nor log segments found. Creating new log.
I20260806 04:58:15.522471 21935 log.cc:824] T 18eaa3f05cb44595b8d9702230c30ab4 P 0c634c0504d446008ab407bb82882031: Log is configured to *not* fsync() on all Append() calls
I20260806 04:58:15.522684 21934 log.cc:824] T 18eaa3f05cb44595b8d9702230c30ab4 P ebc08b93691f474698e93a5cc45baa6c: Log is configured to *not* fsync() on all Append() calls
I20260806 04:58:15.526553 21935 tablet_bootstrap.cc:492] T 18eaa3f05cb44595b8d9702230c30ab4 P 0c634c0504d446008ab407bb82882031: No bootstrap required, opened a new log
I20260806 04:58:15.526840 21935 ts_tablet_manager.cc:1397] T 18eaa3f05cb44595b8d9702230c30ab4 P 0c634c0504d446008ab407bb82882031: Time spent bootstrapping tablet: real 0.011s user 0.006s sys 0.000s
I20260806 04:58:15.526973 21934 tablet_bootstrap.cc:492] T 18eaa3f05cb44595b8d9702230c30ab4 P ebc08b93691f474698e93a5cc45baa6c: No bootstrap required, opened a new log
I20260806 04:58:15.527105 21938 tablet_bootstrap.cc:492] T 18eaa3f05cb44595b8d9702230c30ab4 P 8f45f7b26a8f43799df89071dca99b7a: No bootstrap required, opened a new log
I20260806 04:58:15.527257 21934 ts_tablet_manager.cc:1397] T 18eaa3f05cb44595b8d9702230c30ab4 P ebc08b93691f474698e93a5cc45baa6c: Time spent bootstrapping tablet: real 0.009s user 0.003s sys 0.004s
I20260806 04:58:15.527405 21938 ts_tablet_manager.cc:1397] T 18eaa3f05cb44595b8d9702230c30ab4 P 8f45f7b26a8f43799df89071dca99b7a: Time spent bootstrapping tablet: real 0.012s user 0.009s sys 0.000s
I20260806 04:58:15.536260 21934 raft_consensus.cc:348] T 18eaa3f05cb44595b8d9702230c30ab4 P ebc08b93691f474698e93a5cc45baa6c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f45f7b26a8f43799df89071dca99b7a" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 39027 } } peers { permanent_uuid: "0c634c0504d446008ab407bb82882031" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 46591 } } peers { permanent_uuid: "ebc08b93691f474698e93a5cc45baa6c" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 36799 } }
I20260806 04:58:15.536698 21934 raft_consensus.cc:374] T 18eaa3f05cb44595b8d9702230c30ab4 P ebc08b93691f474698e93a5cc45baa6c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 04:58:15.536520 21938 raft_consensus.cc:348] T 18eaa3f05cb44595b8d9702230c30ab4 P 8f45f7b26a8f43799df89071dca99b7a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f45f7b26a8f43799df89071dca99b7a" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 39027 } } peers { permanent_uuid: "0c634c0504d446008ab407bb82882031" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 46591 } } peers { permanent_uuid: "ebc08b93691f474698e93a5cc45baa6c" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 36799 } }
I20260806 04:58:15.536803 21934 raft_consensus.cc:729] T 18eaa3f05cb44595b8d9702230c30ab4 P ebc08b93691f474698e93a5cc45baa6c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ebc08b93691f474698e93a5cc45baa6c, State: Initialized, Role: FOLLOWER
I20260806 04:58:15.536967 21938 raft_consensus.cc:374] T 18eaa3f05cb44595b8d9702230c30ab4 P 8f45f7b26a8f43799df89071dca99b7a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 04:58:15.537097 21938 raft_consensus.cc:729] T 18eaa3f05cb44595b8d9702230c30ab4 P 8f45f7b26a8f43799df89071dca99b7a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f45f7b26a8f43799df89071dca99b7a, State: Initialized, Role: FOLLOWER
I20260806 04:58:15.537441 21934 consensus_queue.cc:260] T 18eaa3f05cb44595b8d9702230c30ab4 P ebc08b93691f474698e93a5cc45baa6c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f45f7b26a8f43799df89071dca99b7a" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 39027 } } peers { permanent_uuid: "0c634c0504d446008ab407bb82882031" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 46591 } } peers { permanent_uuid: "ebc08b93691f474698e93a5cc45baa6c" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 36799 } }
I20260806 04:58:15.537658 21938 consensus_queue.cc:260] T 18eaa3f05cb44595b8d9702230c30ab4 P 8f45f7b26a8f43799df89071dca99b7a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f45f7b26a8f43799df89071dca99b7a" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 39027 } } peers { permanent_uuid: "0c634c0504d446008ab407bb82882031" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 46591 } } peers { permanent_uuid: "ebc08b93691f474698e93a5cc45baa6c" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 36799 } }
I20260806 04:58:15.538744 21902 heartbeater.cc:503] Master 127.9.96.126:40621 was elected leader, sending a full tablet report...
I20260806 04:58:15.538851 21935 raft_consensus.cc:348] T 18eaa3f05cb44595b8d9702230c30ab4 P 0c634c0504d446008ab407bb82882031 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f45f7b26a8f43799df89071dca99b7a" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 39027 } } peers { permanent_uuid: "0c634c0504d446008ab407bb82882031" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 46591 } } peers { permanent_uuid: "ebc08b93691f474698e93a5cc45baa6c" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 36799 } }
I20260806 04:58:15.539278 21935 raft_consensus.cc:374] T 18eaa3f05cb44595b8d9702230c30ab4 P 0c634c0504d446008ab407bb82882031 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 04:58:15.539413 21934 ts_tablet_manager.cc:1428] T 18eaa3f05cb44595b8d9702230c30ab4 P ebc08b93691f474698e93a5cc45baa6c: Time spent starting tablet: real 0.012s user 0.008s sys 0.004s
I20260806 04:58:15.539407 21935 raft_consensus.cc:729] T 18eaa3f05cb44595b8d9702230c30ab4 P 0c634c0504d446008ab407bb82882031 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0c634c0504d446008ab407bb82882031, State: Initialized, Role: FOLLOWER
I20260806 04:58:15.539992 21935 consensus_queue.cc:260] T 18eaa3f05cb44595b8d9702230c30ab4 P 0c634c0504d446008ab407bb82882031 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f45f7b26a8f43799df89071dca99b7a" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 39027 } } peers { permanent_uuid: "0c634c0504d446008ab407bb82882031" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 46591 } } peers { permanent_uuid: "ebc08b93691f474698e93a5cc45baa6c" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 36799 } }
I20260806 04:58:15.541008 21938 ts_tablet_manager.cc:1428] T 18eaa3f05cb44595b8d9702230c30ab4 P 8f45f7b26a8f43799df89071dca99b7a: Time spent starting tablet: real 0.013s user 0.008s sys 0.005s
I20260806 04:58:15.542512 21695 heartbeater.cc:503] Master 127.9.96.126:40621 was elected leader, sending a full tablet report...
I20260806 04:58:15.543123 21935 ts_tablet_manager.cc:1428] T 18eaa3f05cb44595b8d9702230c30ab4 P 0c634c0504d446008ab407bb82882031: Time spent starting tablet: real 0.016s user 0.007s sys 0.009s
W20260806 04:58:15.596078 21697 tablet.cc:2369] T 18eaa3f05cb44595b8d9702230c30ab4 P 0c634c0504d446008ab407bb82882031: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 04:58:15.614219 21903 tablet.cc:2369] T 18eaa3f05cb44595b8d9702230c30ab4 P ebc08b93691f474698e93a5cc45baa6c: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 04:58:15.748848 21479 tablet.cc:2369] T 18eaa3f05cb44595b8d9702230c30ab4 P 8f45f7b26a8f43799df89071dca99b7a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 04:58:15.872503 21948 raft_consensus.cc:482] T 18eaa3f05cb44595b8d9702230c30ab4 P 0c634c0504d446008ab407bb82882031 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 04:58:15.872843 21948 raft_consensus.cc:504] T 18eaa3f05cb44595b8d9702230c30ab4 P 0c634c0504d446008ab407bb82882031 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f45f7b26a8f43799df89071dca99b7a" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 39027 } } peers { permanent_uuid: "0c634c0504d446008ab407bb82882031" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 46591 } } peers { permanent_uuid: "ebc08b93691f474698e93a5cc45baa6c" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 36799 } }
I20260806 04:58:15.874688 21948 leader_election.cc:288] T 18eaa3f05cb44595b8d9702230c30ab4 P 0c634c0504d446008ab407bb82882031 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8f45f7b26a8f43799df89071dca99b7a (127.9.96.65:39027), ebc08b93691f474698e93a5cc45baa6c (127.9.96.67:36799)
I20260806 04:58:15.881474 21933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:58:15.875088 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.96.66:46331 (local address 127.9.96.67:36799)
0806 04:58:15.875280 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0806 04:58:15.875288 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 04:58:15.876086 (+ 798us) server_negotiation.cc:407] Connection header received
0806 04:58:15.876312 (+ 226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:58:15.876316 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:58:15.876373 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:58:15.876439 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 04:58:15.877078 (+ 639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:15.877789 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:15.879691 (+ 1902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:15.880461 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:15.880557 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:58:15.880681 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0806 04:58:15.881099 (+ 418us) server_negotiation.cc:299] Negotiation successful
0806 04:58:15.881221 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260806 04:58:15.881496 21930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:58:15.875036 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.96.66:36565 (local address 127.9.96.65:39027)
0806 04:58:15.875247 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0806 04:58:15.875255 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 04:58:15.876058 (+ 803us) server_negotiation.cc:407] Connection header received
0806 04:58:15.876293 (+ 235us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:58:15.876303 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:58:15.876375 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:58:15.876489 (+ 114us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 04:58:15.877240 (+ 751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:15.877964 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:15.879739 (+ 1775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:15.880504 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:15.880684 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:58:15.880778 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0806 04:58:15.881146 (+ 368us) server_negotiation.cc:299] Negotiation successful
0806 04:58:15.881260 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260806 04:58:15.881633 21949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:58:15.875029 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.67:36799 (local address 127.9.96.66:46331)
0806 04:58:15.875940 (+ 911us) negotiation.cc:107] Waiting for socket to connect
0806 04:58:15.875963 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0806 04:58:15.876142 (+ 179us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:58:15.876502 (+ 360us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:58:15.876507 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:58:15.876529 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 04:58:15.876915 (+ 386us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:58:15.876929 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:58:15.877947 (+ 1018us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:58:15.877952 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:58:15.879536 (+ 1584us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:58:15.879549 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:58:15.880694 (+ 1145us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:58:15.880700 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:58:15.880885 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:58:15.880940 (+ 55us) client_negotiation.cc:769] Sending connection context
0806 04:58:15.881027 (+ 87us) client_negotiation.cc:240] Negotiation successful
0806 04:58:15.881175 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":733,"mutex_wait_us":25,"spinlock_wait_cycles":14080,"thread_start_us":416,"threads_started":1}
I20260806 04:58:15.881659 21950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:58:15.875094 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.65:39027 (local address 127.9.96.66:36565)
0806 04:58:15.875939 (+ 845us) negotiation.cc:107] Waiting for socket to connect
0806 04:58:15.875955 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 04:58:15.876127 (+ 172us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:58:15.876533 (+ 406us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:58:15.876538 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:58:15.876560 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 04:58:15.877065 (+ 505us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:58:15.877077 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:58:15.878099 (+ 1022us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:58:15.878102 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:58:15.879547 (+ 1445us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:58:15.879560 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:58:15.880785 (+ 1225us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:58:15.880790 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:58:15.880936 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:58:15.880959 (+ 23us) client_negotiation.cc:769] Sending connection context
0806 04:58:15.881089 (+ 130us) client_negotiation.cc:240] Negotiation successful
0806 04:58:15.881208 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":745,"mutex_wait_us":47,"spinlock_wait_cycles":8064,"thread_start_us":441,"threads_started":1}
I20260806 04:58:15.882629 21833 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "18eaa3f05cb44595b8d9702230c30ab4" candidate_uuid: "0c634c0504d446008ab407bb82882031" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ebc08b93691f474698e93a5cc45baa6c" is_pre_election: true
I20260806 04:58:15.882640 21407 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "18eaa3f05cb44595b8d9702230c30ab4" candidate_uuid: "0c634c0504d446008ab407bb82882031" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f45f7b26a8f43799df89071dca99b7a" is_pre_election: true
I20260806 04:58:15.883069 21833 raft_consensus.cc:2454] T 18eaa3f05cb44595b8d9702230c30ab4 P ebc08b93691f474698e93a5cc45baa6c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0c634c0504d446008ab407bb82882031 in term 0.
I20260806 04:58:15.883113 21407 raft_consensus.cc:2454] T 18eaa3f05cb44595b8d9702230c30ab4 P 8f45f7b26a8f43799df89071dca99b7a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0c634c0504d446008ab407bb82882031 in term 0.
I20260806 04:58:15.883723 21509 leader_election.cc:302] T 18eaa3f05cb44595b8d9702230c30ab4 P 0c634c0504d446008ab407bb82882031 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0c634c0504d446008ab407bb82882031, ebc08b93691f474698e93a5cc45baa6c; no voters:
I20260806 04:58:15.884254 21948 raft_consensus.cc:2781] T 18eaa3f05cb44595b8d9702230c30ab4 P 0c634c0504d446008ab407bb82882031 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 04:58:15.884466 21948 raft_consensus.cc:482] T 18eaa3f05cb44595b8d9702230c30ab4 P 0c634c0504d446008ab407bb82882031 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 04:58:15.884590 21948 raft_consensus.cc:3037] T 18eaa3f05cb44595b8d9702230c30ab4 P 0c634c0504d446008ab407bb82882031 [term 0 FOLLOWER]: Advancing to term 1
I20260806 04:58:15.886997 21948 raft_consensus.cc:504] T 18eaa3f05cb44595b8d9702230c30ab4 P 0c634c0504d446008ab407bb82882031 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f45f7b26a8f43799df89071dca99b7a" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 39027 } } peers { permanent_uuid: "0c634c0504d446008ab407bb82882031" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 46591 } } peers { permanent_uuid: "ebc08b93691f474698e93a5cc45baa6c" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 36799 } }
I20260806 04:58:15.887712 21948 leader_election.cc:288] T 18eaa3f05cb44595b8d9702230c30ab4 P 0c634c0504d446008ab407bb82882031 [CANDIDATE]: Term 1 election: Requested vote from peers 8f45f7b26a8f43799df89071dca99b7a (127.9.96.65:39027), ebc08b93691f474698e93a5cc45baa6c (127.9.96.67:36799)
I20260806 04:58:15.888226 21407 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "18eaa3f05cb44595b8d9702230c30ab4" candidate_uuid: "0c634c0504d446008ab407bb82882031" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f45f7b26a8f43799df89071dca99b7a"
I20260806 04:58:15.888285 21833 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "18eaa3f05cb44595b8d9702230c30ab4" candidate_uuid: "0c634c0504d446008ab407bb82882031" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ebc08b93691f474698e93a5cc45baa6c"
I20260806 04:58:15.888434 21407 raft_consensus.cc:3037] T 18eaa3f05cb44595b8d9702230c30ab4 P 8f45f7b26a8f43799df89071dca99b7a [term 0 FOLLOWER]: Advancing to term 1
I20260806 04:58:15.888522 21833 raft_consensus.cc:3037] T 18eaa3f05cb44595b8d9702230c30ab4 P ebc08b93691f474698e93a5cc45baa6c [term 0 FOLLOWER]: Advancing to term 1
I20260806 04:58:15.891460 21833 raft_consensus.cc:2454] T 18eaa3f05cb44595b8d9702230c30ab4 P ebc08b93691f474698e93a5cc45baa6c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0c634c0504d446008ab407bb82882031 in term 1.
I20260806 04:58:15.891705 21407 raft_consensus.cc:2454] T 18eaa3f05cb44595b8d9702230c30ab4 P 8f45f7b26a8f43799df89071dca99b7a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0c634c0504d446008ab407bb82882031 in term 1.
I20260806 04:58:15.892187 21510 leader_election.cc:302] T 18eaa3f05cb44595b8d9702230c30ab4 P 0c634c0504d446008ab407bb82882031 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0c634c0504d446008ab407bb82882031, 8f45f7b26a8f43799df89071dca99b7a; no voters:
I20260806 04:58:15.892537 21948 raft_consensus.cc:2781] T 18eaa3f05cb44595b8d9702230c30ab4 P 0c634c0504d446008ab407bb82882031 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 04:58:15.893126 21948 raft_consensus.cc:686] T 18eaa3f05cb44595b8d9702230c30ab4 P 0c634c0504d446008ab407bb82882031 [term 1 LEADER]: Becoming Leader. State: Replica: 0c634c0504d446008ab407bb82882031, State: Running, Role: LEADER
I20260806 04:58:15.893718 21948 consensus_queue.cc:237] T 18eaa3f05cb44595b8d9702230c30ab4 P 0c634c0504d446008ab407bb82882031 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f45f7b26a8f43799df89071dca99b7a" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 39027 } } peers { permanent_uuid: "0c634c0504d446008ab407bb82882031" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 46591 } } peers { permanent_uuid: "ebc08b93691f474698e93a5cc45baa6c" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 36799 } }
I20260806 04:58:15.900503 21171 catalog_manager.cc:5495] T 18eaa3f05cb44595b8d9702230c30ab4 P 0c634c0504d446008ab407bb82882031 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0c634c0504d446008ab407bb82882031 (127.9.96.66). New cstate: current_term: 1 leader_uuid: "0c634c0504d446008ab407bb82882031" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f45f7b26a8f43799df89071dca99b7a" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 39027 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0c634c0504d446008ab407bb82882031" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 46591 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ebc08b93691f474698e93a5cc45baa6c" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 36799 } health_report { overall_health: UNKNOWN } } }
I20260806 04:58:15.945804 21931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:58:15.940399 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34050 (local address 127.9.96.66:46591)
0806 04:58:15.940621 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0806 04:58:15.940627 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 04:58:15.940646 (+ 19us) server_negotiation.cc:407] Connection header received
0806 04:58:15.940817 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:58:15.940821 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:58:15.940948 (+ 127us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:58:15.941125 (+ 177us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 04:58:15.941629 (+ 504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:15.942385 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:15.943712 (+ 1327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:15.944095 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:15.944188 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:58:15.944618 (+ 430us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 04:58:15.945143 (+ 525us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 04:58:15.945255 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0806 04:58:15.945367 (+ 112us) server_negotiation.cc:299] Negotiation successful
0806 04:58:15.945432 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260806 04:58:15.954389 21833 raft_consensus.cc:1261] T 18eaa3f05cb44595b8d9702230c30ab4 P ebc08b93691f474698e93a5cc45baa6c [term 1 FOLLOWER]: Refusing update from remote peer 0c634c0504d446008ab407bb82882031: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 04:58:15.954376 21407 raft_consensus.cc:1261] T 18eaa3f05cb44595b8d9702230c30ab4 P 8f45f7b26a8f43799df89071dca99b7a [term 1 FOLLOWER]: Refusing update from remote peer 0c634c0504d446008ab407bb82882031: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 04:58:15.955524 21952 consensus_queue.cc:1035] T 18eaa3f05cb44595b8d9702230c30ab4 P 0c634c0504d446008ab407bb82882031 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ebc08b93691f474698e93a5cc45baa6c" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 36799 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 04:58:15.955955 21948 consensus_queue.cc:1035] T 18eaa3f05cb44595b8d9702230c30ab4 P 0c634c0504d446008ab407bb82882031 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f45f7b26a8f43799df89071dca99b7a" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 39027 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 04:58:15.983564 21171 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:57216:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 04:58:15.983884 21171 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:57216:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 04:58:15.987671 21171 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6015b5a9ea39484baf36a8765add32e3: Sending DeleteTablet for 3 replicas of tablet 18eaa3f05cb44595b8d9702230c30ab4
I20260806 04:58:15.988898 21369 tablet_service.cc:1514] Processing DeleteTablet for tablet 18eaa3f05cb44595b8d9702230c30ab4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 04:58:15 UTC) from {username='oryx', principal='oryx/127.9.96.126@KRBTEST.COM'} at 127.0.0.1:35808
I20260806 04:58:15.989611 21802 tablet_service.cc:1514] Processing DeleteTablet for tablet 18eaa3f05cb44595b8d9702230c30ab4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 04:58:15 UTC) from {username='oryx', principal='oryx/127.9.96.126@KRBTEST.COM'} at 127.0.0.1:33152
I20260806 04:58:15.990270 21586 tablet_service.cc:1514] Processing DeleteTablet for tablet 18eaa3f05cb44595b8d9702230c30ab4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 04:58:15 UTC) from {username='oryx', principal='oryx/127.9.96.126@KRBTEST.COM'} at 127.0.0.1:34038
I20260806 04:58:15.990504 21972 tablet_replica.cc:330] T 18eaa3f05cb44595b8d9702230c30ab4 P ebc08b93691f474698e93a5cc45baa6c: stopping tablet replica
I20260806 04:58:15.990914 21972 raft_consensus.cc:2229] T 18eaa3f05cb44595b8d9702230c30ab4 P ebc08b93691f474698e93a5cc45baa6c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 04:58:15.991479 21972 raft_consensus.cc:2258] T 18eaa3f05cb44595b8d9702230c30ab4 P ebc08b93691f474698e93a5cc45baa6c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 04:58:15.993988 21972 ts_tablet_manager.cc:1905] T 18eaa3f05cb44595b8d9702230c30ab4 P ebc08b93691f474698e93a5cc45baa6c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 04:58:15.994371 21171 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:57216:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 04:58:15.995563 21171 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 04:58:16.001653 21972 ts_tablet_manager.cc:1918] T 18eaa3f05cb44595b8d9702230c30ab4 P ebc08b93691f474698e93a5cc45baa6c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 04:58:16.001834 21972 log.cc:1196] T 18eaa3f05cb44595b8d9702230c30ab4 P ebc08b93691f474698e93a5cc45baa6c: Deleting WAL directory at /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785992227050689-9601-0/minicluster-data/ts-2/wal/wals/18eaa3f05cb44595b8d9702230c30ab4
I20260806 04:58:16.002440 21972 ts_tablet_manager.cc:1939] T 18eaa3f05cb44595b8d9702230c30ab4 P ebc08b93691f474698e93a5cc45baa6c: Deleting consensus metadata
I20260806 04:58:16.005358 21585 tablet_service.cc:1467] Processing CreateTablet for tablet cde096dbe74b40deb92026039e59e0eb (DEFAULT_TABLE table=test-table [id=e654629ac5c246be83f6adb8ee45bf8a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 04:58:16.005846 21146 catalog_manager.cc:4841] TS ebc08b93691f474698e93a5cc45baa6c (127.9.96.67:36799): tablet 18eaa3f05cb44595b8d9702230c30ab4 (table test-table [id=abfe56ad98794e8398d938440634754d]) successfully deleted
I20260806 04:58:16.005973 21585 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cde096dbe74b40deb92026039e59e0eb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 04:58:16.006685 21802 tablet_service.cc:1467] Processing CreateTablet for tablet cde096dbe74b40deb92026039e59e0eb (DEFAULT_TABLE table=test-table [id=e654629ac5c246be83f6adb8ee45bf8a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 04:58:16.007207 21802 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cde096dbe74b40deb92026039e59e0eb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 04:58:16.009362 21973 tablet_replica.cc:330] T 18eaa3f05cb44595b8d9702230c30ab4 P 0c634c0504d446008ab407bb82882031: stopping tablet replica
I20260806 04:58:16.009845 21973 raft_consensus.cc:2229] T 18eaa3f05cb44595b8d9702230c30ab4 P 0c634c0504d446008ab407bb82882031 [term 1 LEADER]: Raft consensus shutting down.
I20260806 04:58:16.010674 21973 raft_consensus.cc:2258] T 18eaa3f05cb44595b8d9702230c30ab4 P 0c634c0504d446008ab407bb82882031 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 04:58:16.011871 21368 tablet_service.cc:1467] Processing CreateTablet for tablet cde096dbe74b40deb92026039e59e0eb (DEFAULT_TABLE table=test-table [id=e654629ac5c246be83f6adb8ee45bf8a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 04:58:16.012358 21368 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cde096dbe74b40deb92026039e59e0eb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 04:58:16.012825 21973 ts_tablet_manager.cc:1905] T 18eaa3f05cb44595b8d9702230c30ab4 P 0c634c0504d446008ab407bb82882031: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 04:58:16.013691 21934 tablet_bootstrap.cc:492] T cde096dbe74b40deb92026039e59e0eb P ebc08b93691f474698e93a5cc45baa6c: Bootstrap starting.
I20260806 04:58:16.015129 21934 tablet_bootstrap.cc:654] T cde096dbe74b40deb92026039e59e0eb P ebc08b93691f474698e93a5cc45baa6c: Neither blocks nor log segments found. Creating new log.
I20260806 04:58:16.016376 21968 tablet_replica.cc:330] T 18eaa3f05cb44595b8d9702230c30ab4 P 8f45f7b26a8f43799df89071dca99b7a: stopping tablet replica
I20260806 04:58:16.016963 21968 raft_consensus.cc:2229] T 18eaa3f05cb44595b8d9702230c30ab4 P 8f45f7b26a8f43799df89071dca99b7a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 04:58:16.017490 21968 raft_consensus.cc:2258] T 18eaa3f05cb44595b8d9702230c30ab4 P 8f45f7b26a8f43799df89071dca99b7a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 04:58:16.019505 21938 tablet_bootstrap.cc:492] T cde096dbe74b40deb92026039e59e0eb P 8f45f7b26a8f43799df89071dca99b7a: Bootstrap starting.
I20260806 04:58:16.020579 21935 tablet_bootstrap.cc:492] T cde096dbe74b40deb92026039e59e0eb P 0c634c0504d446008ab407bb82882031: Bootstrap starting.
I20260806 04:58:16.021130 21938 tablet_bootstrap.cc:654] T cde096dbe74b40deb92026039e59e0eb P 8f45f7b26a8f43799df89071dca99b7a: Neither blocks nor log segments found. Creating new log.
I20260806 04:58:16.022833 21935 tablet_bootstrap.cc:654] T cde096dbe74b40deb92026039e59e0eb P 0c634c0504d446008ab407bb82882031: Neither blocks nor log segments found. Creating new log.
I20260806 04:58:16.026036 21935 tablet_bootstrap.cc:492] T cde096dbe74b40deb92026039e59e0eb P 0c634c0504d446008ab407bb82882031: No bootstrap required, opened a new log
I20260806 04:58:16.026162 21968 ts_tablet_manager.cc:1905] T 18eaa3f05cb44595b8d9702230c30ab4 P 8f45f7b26a8f43799df89071dca99b7a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 04:58:16.026202 21935 ts_tablet_manager.cc:1397] T cde096dbe74b40deb92026039e59e0eb P 0c634c0504d446008ab407bb82882031: Time spent bootstrapping tablet: real 0.006s user 0.003s sys 0.001s
I20260806 04:58:16.027062 21935 raft_consensus.cc:348] T cde096dbe74b40deb92026039e59e0eb P 0c634c0504d446008ab407bb82882031 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f45f7b26a8f43799df89071dca99b7a" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 39027 } } peers { permanent_uuid: "0c634c0504d446008ab407bb82882031" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 46591 } } peers { permanent_uuid: "ebc08b93691f474698e93a5cc45baa6c" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 36799 } }
I20260806 04:58:16.027290 21935 raft_consensus.cc:374] T cde096dbe74b40deb92026039e59e0eb P 0c634c0504d446008ab407bb82882031 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 04:58:16.027422 21935 raft_consensus.cc:729] T cde096dbe74b40deb92026039e59e0eb P 0c634c0504d446008ab407bb82882031 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0c634c0504d446008ab407bb82882031, State: Initialized, Role: FOLLOWER
I20260806 04:58:16.027743 21935 consensus_queue.cc:260] T cde096dbe74b40deb92026039e59e0eb P 0c634c0504d446008ab407bb82882031 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f45f7b26a8f43799df89071dca99b7a" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 39027 } } peers { permanent_uuid: "0c634c0504d446008ab407bb82882031" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 46591 } } peers { permanent_uuid: "ebc08b93691f474698e93a5cc45baa6c" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 36799 } }
I20260806 04:58:16.029346 21935 ts_tablet_manager.cc:1428] T cde096dbe74b40deb92026039e59e0eb P 0c634c0504d446008ab407bb82882031: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260806 04:58:16.034937 21968 ts_tablet_manager.cc:1918] T 18eaa3f05cb44595b8d9702230c30ab4 P 8f45f7b26a8f43799df89071dca99b7a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 04:58:16.035192 21968 log.cc:1196] T 18eaa3f05cb44595b8d9702230c30ab4 P 8f45f7b26a8f43799df89071dca99b7a: Deleting WAL directory at /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785992227050689-9601-0/minicluster-data/ts-0/wal/wals/18eaa3f05cb44595b8d9702230c30ab4
I20260806 04:58:16.036118 21968 ts_tablet_manager.cc:1939] T 18eaa3f05cb44595b8d9702230c30ab4 P 8f45f7b26a8f43799df89071dca99b7a: Deleting consensus metadata
I20260806 04:58:16.038333 21148 catalog_manager.cc:4841] TS 8f45f7b26a8f43799df89071dca99b7a (127.9.96.65:39027): tablet 18eaa3f05cb44595b8d9702230c30ab4 (table test-table [id=abfe56ad98794e8398d938440634754d]) successfully deleted
I20260806 04:58:16.040520 21938 tablet_bootstrap.cc:492] T cde096dbe74b40deb92026039e59e0eb P 8f45f7b26a8f43799df89071dca99b7a: No bootstrap required, opened a new log
I20260806 04:58:16.040675 21938 ts_tablet_manager.cc:1397] T cde096dbe74b40deb92026039e59e0eb P 8f45f7b26a8f43799df89071dca99b7a: Time spent bootstrapping tablet: real 0.021s user 0.005s sys 0.007s
I20260806 04:58:16.040717 21934 tablet_bootstrap.cc:492] T cde096dbe74b40deb92026039e59e0eb P ebc08b93691f474698e93a5cc45baa6c: No bootstrap required, opened a new log
I20260806 04:58:16.040882 21934 ts_tablet_manager.cc:1397] T cde096dbe74b40deb92026039e59e0eb P ebc08b93691f474698e93a5cc45baa6c: Time spent bootstrapping tablet: real 0.027s user 0.018s sys 0.003s
I20260806 04:58:16.041420 21938 raft_consensus.cc:348] T cde096dbe74b40deb92026039e59e0eb P 8f45f7b26a8f43799df89071dca99b7a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f45f7b26a8f43799df89071dca99b7a" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 39027 } } peers { permanent_uuid: "0c634c0504d446008ab407bb82882031" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 46591 } } peers { permanent_uuid: "ebc08b93691f474698e93a5cc45baa6c" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 36799 } }
I20260806 04:58:16.041527 21934 raft_consensus.cc:348] T cde096dbe74b40deb92026039e59e0eb P ebc08b93691f474698e93a5cc45baa6c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f45f7b26a8f43799df89071dca99b7a" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 39027 } } peers { permanent_uuid: "0c634c0504d446008ab407bb82882031" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 46591 } } peers { permanent_uuid: "ebc08b93691f474698e93a5cc45baa6c" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 36799 } }
I20260806 04:58:16.041674 21938 raft_consensus.cc:374] T cde096dbe74b40deb92026039e59e0eb P 8f45f7b26a8f43799df89071dca99b7a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 04:58:16.041745 21934 raft_consensus.cc:374] T cde096dbe74b40deb92026039e59e0eb P ebc08b93691f474698e93a5cc45baa6c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 04:58:16.041769 21938 raft_consensus.cc:729] T cde096dbe74b40deb92026039e59e0eb P 8f45f7b26a8f43799df89071dca99b7a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f45f7b26a8f43799df89071dca99b7a, State: Initialized, Role: FOLLOWER
I20260806 04:58:16.041818 21973 ts_tablet_manager.cc:1918] T 18eaa3f05cb44595b8d9702230c30ab4 P 0c634c0504d446008ab407bb82882031: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 04:58:16.041842 21934 raft_consensus.cc:729] T cde096dbe74b40deb92026039e59e0eb P ebc08b93691f474698e93a5cc45baa6c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ebc08b93691f474698e93a5cc45baa6c, State: Initialized, Role: FOLLOWER
I20260806 04:58:16.041981 21973 log.cc:1196] T 18eaa3f05cb44595b8d9702230c30ab4 P 0c634c0504d446008ab407bb82882031: Deleting WAL directory at /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785992227050689-9601-0/minicluster-data/ts-1/wal/wals/18eaa3f05cb44595b8d9702230c30ab4
I20260806 04:58:16.042130 21934 consensus_queue.cc:260] T cde096dbe74b40deb92026039e59e0eb P ebc08b93691f474698e93a5cc45baa6c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f45f7b26a8f43799df89071dca99b7a" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 39027 } } peers { permanent_uuid: "0c634c0504d446008ab407bb82882031" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 46591 } } peers { permanent_uuid: "ebc08b93691f474698e93a5cc45baa6c" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 36799 } }
I20260806 04:58:16.042115 21938 consensus_queue.cc:260] T cde096dbe74b40deb92026039e59e0eb P 8f45f7b26a8f43799df89071dca99b7a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f45f7b26a8f43799df89071dca99b7a" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 39027 } } peers { permanent_uuid: "0c634c0504d446008ab407bb82882031" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 46591 } } peers { permanent_uuid: "ebc08b93691f474698e93a5cc45baa6c" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 36799 } }
I20260806 04:58:16.042526 21973 ts_tablet_manager.cc:1939] T 18eaa3f05cb44595b8d9702230c30ab4 P 0c634c0504d446008ab407bb82882031: Deleting consensus metadata
I20260806 04:58:16.042800 21938 ts_tablet_manager.cc:1428] T cde096dbe74b40deb92026039e59e0eb P 8f45f7b26a8f43799df89071dca99b7a: Time spent starting tablet: real 0.002s user 0.002s sys 0.001s
I20260806 04:58:16.043251 21934 ts_tablet_manager.cc:1428] T cde096dbe74b40deb92026039e59e0eb P ebc08b93691f474698e93a5cc45baa6c: Time spent starting tablet: real 0.002s user 0.002s sys 0.001s
I20260806 04:58:16.044291 21150 catalog_manager.cc:4841] TS 0c634c0504d446008ab407bb82882031 (127.9.96.66:46591): tablet 18eaa3f05cb44595b8d9702230c30ab4 (table test-table [id=abfe56ad98794e8398d938440634754d]) successfully deleted
I20260806 04:58:16.171928 21947 raft_consensus.cc:482] T cde096dbe74b40deb92026039e59e0eb P 8f45f7b26a8f43799df89071dca99b7a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 04:58:16.172199 21947 raft_consensus.cc:504] T cde096dbe74b40deb92026039e59e0eb P 8f45f7b26a8f43799df89071dca99b7a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f45f7b26a8f43799df89071dca99b7a" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 39027 } } peers { permanent_uuid: "0c634c0504d446008ab407bb82882031" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 46591 } } peers { permanent_uuid: "ebc08b93691f474698e93a5cc45baa6c" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 36799 } }
I20260806 04:58:16.174374 21947 leader_election.cc:288] T cde096dbe74b40deb92026039e59e0eb P 8f45f7b26a8f43799df89071dca99b7a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0c634c0504d446008ab407bb82882031 (127.9.96.66:46591), ebc08b93691f474698e93a5cc45baa6c (127.9.96.67:36799)
I20260806 04:58:16.182070 21933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:58:16.175541 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.96.65:60597 (local address 127.9.96.67:36799)
0806 04:58:16.175702 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0806 04:58:16.175707 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 04:58:16.175875 (+ 168us) server_negotiation.cc:407] Connection header received
0806 04:58:16.176078 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:58:16.176083 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:58:16.176177 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:58:16.176273 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 04:58:16.177053 (+ 780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:16.178091 (+ 1038us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:16.180062 (+ 1971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:16.181070 (+ 1008us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:16.181197 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:58:16.181286 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0806 04:58:16.181642 (+ 356us) server_negotiation.cc:299] Negotiation successful
0806 04:58:16.181774 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260806 04:58:16.182116 21981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:58:16.174801 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.67:36799 (local address 127.9.96.65:60597)
0806 04:58:16.175745 (+ 944us) negotiation.cc:107] Waiting for socket to connect
0806 04:58:16.175762 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 04:58:16.175933 (+ 171us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:58:16.176431 (+ 498us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:58:16.176435 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:58:16.176458 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 04:58:16.176893 (+ 435us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:58:16.176904 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:58:16.178250 (+ 1346us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:58:16.178253 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:58:16.179893 (+ 1640us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:58:16.179906 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:58:16.181289 (+ 1383us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:58:16.181296 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:58:16.181460 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:58:16.181494 (+ 34us) client_negotiation.cc:769] Sending connection context
0806 04:58:16.181589 (+ 95us) client_negotiation.cc:240] Negotiation successful
0806 04:58:16.181713 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":625,"mutex_wait_us":4,"thread_start_us":781,"threads_started":1}
I20260806 04:58:16.183048 21833 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cde096dbe74b40deb92026039e59e0eb" candidate_uuid: "8f45f7b26a8f43799df89071dca99b7a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ebc08b93691f474698e93a5cc45baa6c" is_pre_election: true
I20260806 04:58:16.183193 21931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:58:16.175775 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.96.65:38729 (local address 127.9.96.66:46591)
0806 04:58:16.175990 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0806 04:58:16.175997 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 04:58:16.176016 (+ 19us) server_negotiation.cc:407] Connection header received
0806 04:58:16.176084 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:58:16.176088 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:58:16.176149 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:58:16.176248 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 04:58:16.176851 (+ 603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:16.178033 (+ 1182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:16.180638 (+ 2605us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:16.181854 (+ 1216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:16.182029 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:58:16.182140 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0806 04:58:16.182712 (+ 572us) server_negotiation.cc:299] Negotiation successful
0806 04:58:16.182881 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260806 04:58:16.183329 21833 raft_consensus.cc:2454] T cde096dbe74b40deb92026039e59e0eb P ebc08b93691f474698e93a5cc45baa6c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f45f7b26a8f43799df89071dca99b7a in term 0.
I20260806 04:58:16.183780 21978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:58:16.174536 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.66:46591 (local address 127.9.96.65:38729)
0806 04:58:16.175298 (+ 762us) negotiation.cc:107] Waiting for socket to connect
0806 04:58:16.175323 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0806 04:58:16.175504 (+ 181us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:58:16.176277 (+ 773us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:58:16.176281 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:58:16.176303 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 04:58:16.176668 (+ 365us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:58:16.176677 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:58:16.178223 (+ 1546us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:58:16.178228 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:58:16.180466 (+ 2238us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:58:16.180480 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:58:16.182329 (+ 1849us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:58:16.182337 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:58:16.182521 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:58:16.182571 (+ 50us) client_negotiation.cc:769] Sending connection context
0806 04:58:16.182664 (+ 93us) client_negotiation.cc:240] Negotiation successful
0806 04:58:16.182785 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":607,"thread_start_us":418,"threads_started":1}
I20260806 04:58:16.183868 21293 leader_election.cc:302] T cde096dbe74b40deb92026039e59e0eb P 8f45f7b26a8f43799df89071dca99b7a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8f45f7b26a8f43799df89071dca99b7a, ebc08b93691f474698e93a5cc45baa6c; no voters:
I20260806 04:58:16.184293 21947 raft_consensus.cc:2781] T cde096dbe74b40deb92026039e59e0eb P 8f45f7b26a8f43799df89071dca99b7a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 04:58:16.184458 21947 raft_consensus.cc:482] T cde096dbe74b40deb92026039e59e0eb P 8f45f7b26a8f43799df89071dca99b7a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 04:58:16.184543 21947 raft_consensus.cc:3037] T cde096dbe74b40deb92026039e59e0eb P 8f45f7b26a8f43799df89071dca99b7a [term 0 FOLLOWER]: Advancing to term 1
I20260806 04:58:16.184983 21623 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cde096dbe74b40deb92026039e59e0eb" candidate_uuid: "8f45f7b26a8f43799df89071dca99b7a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0c634c0504d446008ab407bb82882031" is_pre_election: true
I20260806 04:58:16.185421 21623 raft_consensus.cc:2454] T cde096dbe74b40deb92026039e59e0eb P 0c634c0504d446008ab407bb82882031 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f45f7b26a8f43799df89071dca99b7a in term 0.
I20260806 04:58:16.187719 21947 raft_consensus.cc:504] T cde096dbe74b40deb92026039e59e0eb P 8f45f7b26a8f43799df89071dca99b7a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f45f7b26a8f43799df89071dca99b7a" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 39027 } } peers { permanent_uuid: "0c634c0504d446008ab407bb82882031" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 46591 } } peers { permanent_uuid: "ebc08b93691f474698e93a5cc45baa6c" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 36799 } }
I20260806 04:58:16.188417 21947 leader_election.cc:288] T cde096dbe74b40deb92026039e59e0eb P 8f45f7b26a8f43799df89071dca99b7a [CANDIDATE]: Term 1 election: Requested vote from peers 0c634c0504d446008ab407bb82882031 (127.9.96.66:46591), ebc08b93691f474698e93a5cc45baa6c (127.9.96.67:36799)
I20260806 04:58:16.188871 21623 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cde096dbe74b40deb92026039e59e0eb" candidate_uuid: "8f45f7b26a8f43799df89071dca99b7a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0c634c0504d446008ab407bb82882031"
I20260806 04:58:16.188923 21833 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cde096dbe74b40deb92026039e59e0eb" candidate_uuid: "8f45f7b26a8f43799df89071dca99b7a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ebc08b93691f474698e93a5cc45baa6c"
I20260806 04:58:16.189061 21623 raft_consensus.cc:3037] T cde096dbe74b40deb92026039e59e0eb P 0c634c0504d446008ab407bb82882031 [term 0 FOLLOWER]: Advancing to term 1
I20260806 04:58:16.189178 21833 raft_consensus.cc:3037] T cde096dbe74b40deb92026039e59e0eb P ebc08b93691f474698e93a5cc45baa6c [term 0 FOLLOWER]: Advancing to term 1
I20260806 04:58:16.191304 21833 raft_consensus.cc:2454] T cde096dbe74b40deb92026039e59e0eb P ebc08b93691f474698e93a5cc45baa6c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f45f7b26a8f43799df89071dca99b7a in term 1.
I20260806 04:58:16.191776 21293 leader_election.cc:302] T cde096dbe74b40deb92026039e59e0eb P 8f45f7b26a8f43799df89071dca99b7a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8f45f7b26a8f43799df89071dca99b7a, ebc08b93691f474698e93a5cc45baa6c; no voters:
I20260806 04:58:16.192096 21947 raft_consensus.cc:2781] T cde096dbe74b40deb92026039e59e0eb P 8f45f7b26a8f43799df89071dca99b7a [term 1 FOLLOWER]: Leader election won for term 1
I20260806 04:58:16.192200 21623 raft_consensus.cc:2454] T cde096dbe74b40deb92026039e59e0eb P 0c634c0504d446008ab407bb82882031 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f45f7b26a8f43799df89071dca99b7a in term 1.
I20260806 04:58:16.192827 21947 raft_consensus.cc:686] T cde096dbe74b40deb92026039e59e0eb P 8f45f7b26a8f43799df89071dca99b7a [term 1 LEADER]: Becoming Leader. State: Replica: 8f45f7b26a8f43799df89071dca99b7a, State: Running, Role: LEADER
I20260806 04:58:16.193249 21947 consensus_queue.cc:237] T cde096dbe74b40deb92026039e59e0eb P 8f45f7b26a8f43799df89071dca99b7a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f45f7b26a8f43799df89071dca99b7a" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 39027 } } peers { permanent_uuid: "0c634c0504d446008ab407bb82882031" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 46591 } } peers { permanent_uuid: "ebc08b93691f474698e93a5cc45baa6c" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 36799 } }
I20260806 04:58:16.196816 21169 catalog_manager.cc:5495] T cde096dbe74b40deb92026039e59e0eb P 8f45f7b26a8f43799df89071dca99b7a reported cstate change: term changed from 0 to 1, leader changed from <none> to 8f45f7b26a8f43799df89071dca99b7a (127.9.96.65). New cstate: current_term: 1 leader_uuid: "8f45f7b26a8f43799df89071dca99b7a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f45f7b26a8f43799df89071dca99b7a" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 39027 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0c634c0504d446008ab407bb82882031" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 46591 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ebc08b93691f474698e93a5cc45baa6c" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 36799 } health_report { overall_health: UNKNOWN } } }
Aug 06 04:58:16 dist-test-slave-x4qh krb5kdc[21106](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992293, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.96.126@KRBTEST.COM for oryx/127.9.96.126@KRBTEST.COM
I20260806 04:58:16.267889 21998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:58:16.246080 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:40621 (local address 127.0.0.1:57220)
0806 04:58:16.253327 (+ 7247us) negotiation.cc:107] Waiting for socket to connect
0806 04:58:16.253359 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0806 04:58:16.253549 (+ 190us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:58:16.254662 (+ 1113us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:58:16.254666 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:58:16.255447 (+ 781us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:58:16.255676 (+ 229us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:58:16.255685 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:58:16.256765 (+ 1080us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:58:16.256771 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:58:16.257913 (+ 1142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:58:16.257926 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:58:16.258688 (+ 762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:58:16.258694 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:58:16.258893 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:58:16.259621 (+ 728us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:58:16.259638 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:58:16.262032 (+ 2394us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 04:58:16.265270 (+ 3238us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:58:16.265285 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:58:16.265292 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:58:16.265713 (+ 421us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:58:16.266078 (+ 365us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:58:16.266084 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:58:16.266087 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:58:16.266226 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:58:16.266758 (+ 532us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 04:58:16.266766 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 04:58:16.266935 (+ 169us) client_negotiation.cc:769] Sending connection context
0806 04:58:16.267120 (+ 185us) client_negotiation.cc:240] Negotiation successful
0806 04:58:16.267433 (+ 313us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7037,"thread_start_us":7575,"threads_started":1}
I20260806 04:58:16.267908 21997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:58:16.246070 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57220 (local address 127.9.96.126:40621)
0806 04:58:16.254369 (+ 8299us) server_negotiation.cc:206] Beginning negotiation
0806 04:58:16.254373 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 04:58:16.254395 (+ 22us) server_negotiation.cc:407] Connection header received
0806 04:58:16.254439 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:58:16.254443 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:58:16.254498 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:58:16.254635 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:58:16.255846 (+ 1211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:16.256610 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:16.258067 (+ 1457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:16.258505 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:16.258654 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:58:16.262246 (+ 3592us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:58:16.262251 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:58:16.262255 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:58:16.262282 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:58:16.265117 (+ 2835us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:58:16.265846 (+ 729us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:58:16.265851 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:58:16.265854 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:58:16.265967 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:58:16.266345 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:58:16.266350 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:58:16.266353 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:58:16.266625 (+ 272us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:58:16.266780 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0806 04:58:16.267213 (+ 433us) server_negotiation.cc:299] Negotiation successful
0806 04:58:16.267557 (+ 344us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":154,"server-negotiator.queue_time_us":8160,"thread_start_us":8022,"threads_started":1}
I20260806 04:58:16.273556 21169 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.9.96.126@KRBTEST.COM'} at 127.0.0.1:57220:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260806 04:58:16.274739 21169 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 04:58:16.281888 21368 tablet_service.cc:1467] Processing CreateTablet for tablet 781b6631a658446db1b07093b7b6c644 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=d6be1f063b8b41db8ecfb0c544a9dda3]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 04:58:16.282500 21368 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 781b6631a658446db1b07093b7b6c644. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 04:58:16.282248 21802 tablet_service.cc:1467] Processing CreateTablet for tablet 781b6631a658446db1b07093b7b6c644 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=d6be1f063b8b41db8ecfb0c544a9dda3]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 04:58:16.283000 21802 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 781b6631a658446db1b07093b7b6c644. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 04:58:16.287979 21585 tablet_service.cc:1467] Processing CreateTablet for tablet 781b6631a658446db1b07093b7b6c644 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=d6be1f063b8b41db8ecfb0c544a9dda3]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 04:58:16.288728 21585 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 781b6631a658446db1b07093b7b6c644. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 04:58:16.289954 21938 tablet_bootstrap.cc:492] T 781b6631a658446db1b07093b7b6c644 P 8f45f7b26a8f43799df89071dca99b7a: Bootstrap starting.
I20260806 04:58:16.290510 21934 tablet_bootstrap.cc:492] T 781b6631a658446db1b07093b7b6c644 P ebc08b93691f474698e93a5cc45baa6c: Bootstrap starting.
I20260806 04:58:16.291733 21938 tablet_bootstrap.cc:654] T 781b6631a658446db1b07093b7b6c644 P 8f45f7b26a8f43799df89071dca99b7a: Neither blocks nor log segments found. Creating new log.
I20260806 04:58:16.292076 21934 tablet_bootstrap.cc:654] T 781b6631a658446db1b07093b7b6c644 P ebc08b93691f474698e93a5cc45baa6c: Neither blocks nor log segments found. Creating new log.
I20260806 04:58:16.295086 21938 tablet_bootstrap.cc:492] T 781b6631a658446db1b07093b7b6c644 P 8f45f7b26a8f43799df89071dca99b7a: No bootstrap required, opened a new log
I20260806 04:58:16.295269 21938 ts_tablet_manager.cc:1397] T 781b6631a658446db1b07093b7b6c644 P 8f45f7b26a8f43799df89071dca99b7a: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260806 04:58:16.296068 21938 raft_consensus.cc:348] T 781b6631a658446db1b07093b7b6c644 P 8f45f7b26a8f43799df89071dca99b7a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f45f7b26a8f43799df89071dca99b7a" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 39027 } } peers { permanent_uuid: "0c634c0504d446008ab407bb82882031" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 46591 } } peers { permanent_uuid: "ebc08b93691f474698e93a5cc45baa6c" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 36799 } }
I20260806 04:58:16.296455 21938 raft_consensus.cc:374] T 781b6631a658446db1b07093b7b6c644 P 8f45f7b26a8f43799df89071dca99b7a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 04:58:16.296558 21938 raft_consensus.cc:729] T 781b6631a658446db1b07093b7b6c644 P 8f45f7b26a8f43799df89071dca99b7a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f45f7b26a8f43799df89071dca99b7a, State: Initialized, Role: FOLLOWER
I20260806 04:58:16.297869 21938 consensus_queue.cc:260] T 781b6631a658446db1b07093b7b6c644 P 8f45f7b26a8f43799df89071dca99b7a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f45f7b26a8f43799df89071dca99b7a" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 39027 } } peers { permanent_uuid: "0c634c0504d446008ab407bb82882031" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 46591 } } peers { permanent_uuid: "ebc08b93691f474698e93a5cc45baa6c" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 36799 } }
I20260806 04:58:16.298408 21947 tablet_replica.cc:439] T 781b6631a658446db1b07093b7b6c644 P 8f45f7b26a8f43799df89071dca99b7a: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f45f7b26a8f43799df89071dca99b7a" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 39027 } } peers { permanent_uuid: "0c634c0504d446008ab407bb82882031" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 46591 } } peers { permanent_uuid: "ebc08b93691f474698e93a5cc45baa6c" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 36799 } } }
I20260806 04:58:16.299409 21947 tablet_replica.cc:442] T 781b6631a658446db1b07093b7b6c644 P 8f45f7b26a8f43799df89071dca99b7a: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 04:58:16.298621 21938 ts_tablet_manager.cc:1428] T 781b6631a658446db1b07093b7b6c644 P 8f45f7b26a8f43799df89071dca99b7a: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 04:58:16.302012 21935 tablet_bootstrap.cc:492] T 781b6631a658446db1b07093b7b6c644 P 0c634c0504d446008ab407bb82882031: Bootstrap starting.
I20260806 04:58:16.302366 21934 tablet_bootstrap.cc:492] T 781b6631a658446db1b07093b7b6c644 P ebc08b93691f474698e93a5cc45baa6c: No bootstrap required, opened a new log
I20260806 04:58:16.302562 21934 ts_tablet_manager.cc:1397] T 781b6631a658446db1b07093b7b6c644 P ebc08b93691f474698e93a5cc45baa6c: Time spent bootstrapping tablet: real 0.012s user 0.010s sys 0.000s
I20260806 04:58:16.303511 21934 raft_consensus.cc:348] T 781b6631a658446db1b07093b7b6c644 P ebc08b93691f474698e93a5cc45baa6c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f45f7b26a8f43799df89071dca99b7a" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 39027 } } peers { permanent_uuid: "0c634c0504d446008ab407bb82882031" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 46591 } } peers { permanent_uuid: "ebc08b93691f474698e93a5cc45baa6c" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 36799 } }
I20260806 04:58:16.303809 21934 raft_consensus.cc:374] T 781b6631a658446db1b07093b7b6c644 P ebc08b93691f474698e93a5cc45baa6c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 04:58:16.303902 21934 raft_consensus.cc:729] T 781b6631a658446db1b07093b7b6c644 P ebc08b93691f474698e93a5cc45baa6c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ebc08b93691f474698e93a5cc45baa6c, State: Initialized, Role: FOLLOWER
I20260806 04:58:16.304199 21935 tablet_bootstrap.cc:654] T 781b6631a658446db1b07093b7b6c644 P 0c634c0504d446008ab407bb82882031: Neither blocks nor log segments found. Creating new log.
I20260806 04:58:16.304193 21934 consensus_queue.cc:260] T 781b6631a658446db1b07093b7b6c644 P ebc08b93691f474698e93a5cc45baa6c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f45f7b26a8f43799df89071dca99b7a" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 39027 } } peers { permanent_uuid: "0c634c0504d446008ab407bb82882031" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 46591 } } peers { permanent_uuid: "ebc08b93691f474698e93a5cc45baa6c" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 36799 } }
I20260806 04:58:16.304899 21934 ts_tablet_manager.cc:1428] T 781b6631a658446db1b07093b7b6c644 P ebc08b93691f474698e93a5cc45baa6c: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260806 04:58:16.304797 21946 tablet_replica.cc:439] T 781b6631a658446db1b07093b7b6c644 P ebc08b93691f474698e93a5cc45baa6c: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f45f7b26a8f43799df89071dca99b7a" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 39027 } } peers { permanent_uuid: "0c634c0504d446008ab407bb82882031" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 46591 } } peers { permanent_uuid: "ebc08b93691f474698e93a5cc45baa6c" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 36799 } } }
I20260806 04:58:16.305193 21946 tablet_replica.cc:442] T 781b6631a658446db1b07093b7b6c644 P ebc08b93691f474698e93a5cc45baa6c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 04:58:16.307680 21935 tablet_bootstrap.cc:492] T 781b6631a658446db1b07093b7b6c644 P 0c634c0504d446008ab407bb82882031: No bootstrap required, opened a new log
I20260806 04:58:16.308014 21935 ts_tablet_manager.cc:1397] T 781b6631a658446db1b07093b7b6c644 P 0c634c0504d446008ab407bb82882031: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.001s
I20260806 04:58:16.308681 21935 raft_consensus.cc:348] T 781b6631a658446db1b07093b7b6c644 P 0c634c0504d446008ab407bb82882031 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f45f7b26a8f43799df89071dca99b7a" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 39027 } } peers { permanent_uuid: "0c634c0504d446008ab407bb82882031" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 46591 } } peers { permanent_uuid: "ebc08b93691f474698e93a5cc45baa6c" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 36799 } }
I20260806 04:58:16.308931 21935 raft_consensus.cc:374] T 781b6631a658446db1b07093b7b6c644 P 0c634c0504d446008ab407bb82882031 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 04:58:16.309053 21935 raft_consensus.cc:729] T 781b6631a658446db1b07093b7b6c644 P 0c634c0504d446008ab407bb82882031 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0c634c0504d446008ab407bb82882031, State: Initialized, Role: FOLLOWER
I20260806 04:58:16.309348 21935 consensus_queue.cc:260] T 781b6631a658446db1b07093b7b6c644 P 0c634c0504d446008ab407bb82882031 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f45f7b26a8f43799df89071dca99b7a" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 39027 } } peers { permanent_uuid: "0c634c0504d446008ab407bb82882031" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 46591 } } peers { permanent_uuid: "ebc08b93691f474698e93a5cc45baa6c" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 36799 } }
I20260806 04:58:16.309798 21948 tablet_replica.cc:439] T 781b6631a658446db1b07093b7b6c644 P 0c634c0504d446008ab407bb82882031: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f45f7b26a8f43799df89071dca99b7a" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 39027 } } peers { permanent_uuid: "0c634c0504d446008ab407bb82882031" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 46591 } } peers { permanent_uuid: "ebc08b93691f474698e93a5cc45baa6c" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 36799 } } }
I20260806 04:58:16.310097 21948 tablet_replica.cc:442] T 781b6631a658446db1b07093b7b6c644 P 0c634c0504d446008ab407bb82882031: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 04:58:16.309964 21935 ts_tablet_manager.cc:1428] T 781b6631a658446db1b07093b7b6c644 P 0c634c0504d446008ab407bb82882031: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260806 04:58:16.407298 21947 raft_consensus.cc:482] T 781b6631a658446db1b07093b7b6c644 P 8f45f7b26a8f43799df89071dca99b7a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 04:58:16.407574 21947 raft_consensus.cc:504] T 781b6631a658446db1b07093b7b6c644 P 8f45f7b26a8f43799df89071dca99b7a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f45f7b26a8f43799df89071dca99b7a" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 39027 } } peers { permanent_uuid: "0c634c0504d446008ab407bb82882031" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 46591 } } peers { permanent_uuid: "ebc08b93691f474698e93a5cc45baa6c" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 36799 } }
I20260806 04:58:16.408253 21947 leader_election.cc:288] T 781b6631a658446db1b07093b7b6c644 P 8f45f7b26a8f43799df89071dca99b7a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0c634c0504d446008ab407bb82882031 (127.9.96.66:46591), ebc08b93691f474698e93a5cc45baa6c (127.9.96.67:36799)
I20260806 04:58:16.408830 21623 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "781b6631a658446db1b07093b7b6c644" candidate_uuid: "8f45f7b26a8f43799df89071dca99b7a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0c634c0504d446008ab407bb82882031" is_pre_election: true
I20260806 04:58:16.408921 21833 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "781b6631a658446db1b07093b7b6c644" candidate_uuid: "8f45f7b26a8f43799df89071dca99b7a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ebc08b93691f474698e93a5cc45baa6c" is_pre_election: true
I20260806 04:58:16.409109 21623 raft_consensus.cc:2454] T 781b6631a658446db1b07093b7b6c644 P 0c634c0504d446008ab407bb82882031 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f45f7b26a8f43799df89071dca99b7a in term 0.
I20260806 04:58:16.409181 21833 raft_consensus.cc:2454] T 781b6631a658446db1b07093b7b6c644 P ebc08b93691f474698e93a5cc45baa6c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f45f7b26a8f43799df89071dca99b7a in term 0.
I20260806 04:58:16.409603 21298 leader_election.cc:302] T 781b6631a658446db1b07093b7b6c644 P 8f45f7b26a8f43799df89071dca99b7a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0c634c0504d446008ab407bb82882031, 8f45f7b26a8f43799df89071dca99b7a; no voters:
I20260806 04:58:16.409920 21947 raft_consensus.cc:2781] T 781b6631a658446db1b07093b7b6c644 P 8f45f7b26a8f43799df89071dca99b7a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 04:58:16.410032 21947 raft_consensus.cc:482] T 781b6631a658446db1b07093b7b6c644 P 8f45f7b26a8f43799df89071dca99b7a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 04:58:16.410141 21947 raft_consensus.cc:3037] T 781b6631a658446db1b07093b7b6c644 P 8f45f7b26a8f43799df89071dca99b7a [term 0 FOLLOWER]: Advancing to term 1
I20260806 04:58:16.413174 21947 raft_consensus.cc:504] T 781b6631a658446db1b07093b7b6c644 P 8f45f7b26a8f43799df89071dca99b7a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f45f7b26a8f43799df89071dca99b7a" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 39027 } } peers { permanent_uuid: "0c634c0504d446008ab407bb82882031" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 46591 } } peers { permanent_uuid: "ebc08b93691f474698e93a5cc45baa6c" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 36799 } }
I20260806 04:58:16.413837 21947 leader_election.cc:288] T 781b6631a658446db1b07093b7b6c644 P 8f45f7b26a8f43799df89071dca99b7a [CANDIDATE]: Term 1 election: Requested vote from peers 0c634c0504d446008ab407bb82882031 (127.9.96.66:46591), ebc08b93691f474698e93a5cc45baa6c (127.9.96.67:36799)
I20260806 04:58:16.414266 21833 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "781b6631a658446db1b07093b7b6c644" candidate_uuid: "8f45f7b26a8f43799df89071dca99b7a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ebc08b93691f474698e93a5cc45baa6c"
I20260806 04:58:16.414305 21623 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "781b6631a658446db1b07093b7b6c644" candidate_uuid: "8f45f7b26a8f43799df89071dca99b7a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0c634c0504d446008ab407bb82882031"
I20260806 04:58:16.414515 21833 raft_consensus.cc:3037] T 781b6631a658446db1b07093b7b6c644 P ebc08b93691f474698e93a5cc45baa6c [term 0 FOLLOWER]: Advancing to term 1
I20260806 04:58:16.414561 21623 raft_consensus.cc:3037] T 781b6631a658446db1b07093b7b6c644 P 0c634c0504d446008ab407bb82882031 [term 0 FOLLOWER]: Advancing to term 1
I20260806 04:58:16.417160 21833 raft_consensus.cc:2454] T 781b6631a658446db1b07093b7b6c644 P ebc08b93691f474698e93a5cc45baa6c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f45f7b26a8f43799df89071dca99b7a in term 1.
I20260806 04:58:16.417580 21293 leader_election.cc:302] T 781b6631a658446db1b07093b7b6c644 P 8f45f7b26a8f43799df89071dca99b7a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8f45f7b26a8f43799df89071dca99b7a, ebc08b93691f474698e93a5cc45baa6c; no voters:
I20260806 04:58:16.417838 21947 raft_consensus.cc:2781] T 781b6631a658446db1b07093b7b6c644 P 8f45f7b26a8f43799df89071dca99b7a [term 1 FOLLOWER]: Leader election won for term 1
I20260806 04:58:16.417814 21623 raft_consensus.cc:2454] T 781b6631a658446db1b07093b7b6c644 P 0c634c0504d446008ab407bb82882031 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f45f7b26a8f43799df89071dca99b7a in term 1.
I20260806 04:58:16.418028 21947 raft_consensus.cc:686] T 781b6631a658446db1b07093b7b6c644 P 8f45f7b26a8f43799df89071dca99b7a [term 1 LEADER]: Becoming Leader. State: Replica: 8f45f7b26a8f43799df89071dca99b7a, State: Running, Role: LEADER
I20260806 04:58:16.418273 21947 consensus_queue.cc:237] T 781b6631a658446db1b07093b7b6c644 P 8f45f7b26a8f43799df89071dca99b7a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f45f7b26a8f43799df89071dca99b7a" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 39027 } } peers { permanent_uuid: "0c634c0504d446008ab407bb82882031" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 46591 } } peers { permanent_uuid: "ebc08b93691f474698e93a5cc45baa6c" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 36799 } }
I20260806 04:58:16.419898 21984 tablet_replica.cc:439] T 781b6631a658446db1b07093b7b6c644 P 8f45f7b26a8f43799df89071dca99b7a: TxnStatusTablet state changed. Reason: New leader 8f45f7b26a8f43799df89071dca99b7a. Latest consensus state: current_term: 1 leader_uuid: "8f45f7b26a8f43799df89071dca99b7a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f45f7b26a8f43799df89071dca99b7a" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 39027 } } peers { permanent_uuid: "0c634c0504d446008ab407bb82882031" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 46591 } } peers { permanent_uuid: "ebc08b93691f474698e93a5cc45baa6c" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 36799 } } }
I20260806 04:58:16.420245 21984 tablet_replica.cc:442] T 781b6631a658446db1b07093b7b6c644 P 8f45f7b26a8f43799df89071dca99b7a: This TxnStatusTablet replica's current role is: LEADER
I20260806 04:58:16.420907 22005 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260806 04:58:16.421157 22005 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260806 04:58:16.421630 21170 catalog_manager.cc:5495] T 781b6631a658446db1b07093b7b6c644 P 8f45f7b26a8f43799df89071dca99b7a reported cstate change: term changed from 0 to 1, leader changed from <none> to 8f45f7b26a8f43799df89071dca99b7a (127.9.96.65). New cstate: current_term: 1 leader_uuid: "8f45f7b26a8f43799df89071dca99b7a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f45f7b26a8f43799df89071dca99b7a" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 39027 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0c634c0504d446008ab407bb82882031" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 46591 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ebc08b93691f474698e93a5cc45baa6c" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 36799 } health_report { overall_health: UNKNOWN } } }
I20260806 04:58:16.457979 21998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:58:16.450546 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.65:39027 (local address 127.0.0.1:35810)
0806 04:58:16.450957 (+ 411us) negotiation.cc:107] Waiting for socket to connect
0806 04:58:16.450978 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0806 04:58:16.451113 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:58:16.452274 (+ 1161us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:58:16.452280 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:58:16.452294 (+ 14us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 04:58:16.452498 (+ 204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:58:16.452510 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:58:16.454098 (+ 1588us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:58:16.454102 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:58:16.455109 (+ 1007us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:58:16.455120 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:58:16.456088 (+ 968us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:58:16.456092 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:58:16.456242 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:58:16.456296 (+ 54us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 04:58:16.457451 (+ 1155us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 04:58:16.457477 (+ 26us) client_negotiation.cc:769] Sending connection context
0806 04:58:16.457574 (+ 97us) client_negotiation.cc:240] Negotiation successful
0806 04:58:16.457727 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":242}
I20260806 04:58:16.458325 22010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:58:16.450495 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35810 (local address 127.9.96.65:39027)
0806 04:58:16.451798 (+ 1303us) server_negotiation.cc:206] Beginning negotiation
0806 04:58:16.451804 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 04:58:16.451831 (+ 27us) server_negotiation.cc:407] Connection header received
0806 04:58:16.451924 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:58:16.451931 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:58:16.452046 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:58:16.452214 (+ 168us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 04:58:16.452689 (+ 475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:16.453956 (+ 1267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:16.455259 (+ 1303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:16.455942 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:16.456070 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:58:16.456534 (+ 464us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 04:58:16.457316 (+ 782us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 04:58:16.457478 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0806 04:58:16.457641 (+ 163us) server_negotiation.cc:299] Negotiation successful
0806 04:58:16.457725 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1166,"thread_start_us":970,"threads_started":1}
I20260806 04:58:16.462495 21833 raft_consensus.cc:1261] T 781b6631a658446db1b07093b7b6c644 P ebc08b93691f474698e93a5cc45baa6c [term 1 FOLLOWER]: Refusing update from remote peer 8f45f7b26a8f43799df89071dca99b7a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 04:58:16.462679 21623 raft_consensus.cc:1261] T 781b6631a658446db1b07093b7b6c644 P 0c634c0504d446008ab407bb82882031 [term 1 FOLLOWER]: Refusing update from remote peer 8f45f7b26a8f43799df89071dca99b7a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 04:58:16.463294 21984 consensus_queue.cc:1035] T 781b6631a658446db1b07093b7b6c644 P 8f45f7b26a8f43799df89071dca99b7a [LEADER]: Connected to new peer: Peer: permanent_uuid: "ebc08b93691f474698e93a5cc45baa6c" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 36799 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 04:58:16.463728 21947 consensus_queue.cc:1035] T 781b6631a658446db1b07093b7b6c644 P 8f45f7b26a8f43799df89071dca99b7a [LEADER]: Connected to new peer: Peer: permanent_uuid: "0c634c0504d446008ab407bb82882031" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 46591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 04:58:16.467604 21948 tablet_replica.cc:439] T 781b6631a658446db1b07093b7b6c644 P 0c634c0504d446008ab407bb82882031: TxnStatusTablet state changed. Reason: New leader 8f45f7b26a8f43799df89071dca99b7a. Latest consensus state: current_term: 1 leader_uuid: "8f45f7b26a8f43799df89071dca99b7a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f45f7b26a8f43799df89071dca99b7a" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 39027 } } peers { permanent_uuid: "0c634c0504d446008ab407bb82882031" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 46591 } } peers { permanent_uuid: "ebc08b93691f474698e93a5cc45baa6c" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 36799 } } }
I20260806 04:58:16.467978 21948 tablet_replica.cc:442] T 781b6631a658446db1b07093b7b6c644 P 0c634c0504d446008ab407bb82882031: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 04:58:16.471151 21947 tablet_replica.cc:439] T 781b6631a658446db1b07093b7b6c644 P 8f45f7b26a8f43799df89071dca99b7a: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "8f45f7b26a8f43799df89071dca99b7a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f45f7b26a8f43799df89071dca99b7a" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 39027 } } peers { permanent_uuid: "0c634c0504d446008ab407bb82882031" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 46591 } } peers { permanent_uuid: "ebc08b93691f474698e93a5cc45baa6c" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 36799 } } }
I20260806 04:58:16.471505 21947 tablet_replica.cc:442] T 781b6631a658446db1b07093b7b6c644 P 8f45f7b26a8f43799df89071dca99b7a: This TxnStatusTablet replica's current role is: LEADER
I20260806 04:58:16.472059 21952 tablet_replica.cc:439] T 781b6631a658446db1b07093b7b6c644 P 0c634c0504d446008ab407bb82882031: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "8f45f7b26a8f43799df89071dca99b7a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f45f7b26a8f43799df89071dca99b7a" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 39027 } } peers { permanent_uuid: "0c634c0504d446008ab407bb82882031" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 46591 } } peers { permanent_uuid: "ebc08b93691f474698e93a5cc45baa6c" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 36799 } } }
I20260806 04:58:16.472141 21946 tablet_replica.cc:439] T 781b6631a658446db1b07093b7b6c644 P ebc08b93691f474698e93a5cc45baa6c: TxnStatusTablet state changed. Reason: New leader 8f45f7b26a8f43799df89071dca99b7a. Latest consensus state: current_term: 1 leader_uuid: "8f45f7b26a8f43799df89071dca99b7a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f45f7b26a8f43799df89071dca99b7a" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 39027 } } peers { permanent_uuid: "0c634c0504d446008ab407bb82882031" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 46591 } } peers { permanent_uuid: "ebc08b93691f474698e93a5cc45baa6c" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 36799 } } }
I20260806 04:58:16.472380 21952 tablet_replica.cc:442] T 781b6631a658446db1b07093b7b6c644 P 0c634c0504d446008ab407bb82882031: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 04:58:16.472434 21946 tablet_replica.cc:442] T 781b6631a658446db1b07093b7b6c644 P ebc08b93691f474698e93a5cc45baa6c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 04:58:16.472369 21955 mvcc.cc:205] Tried to move back new op lower bound from 7315424446302937088 to 7315424446131363840. Current Snapshot: MvccSnapshot[applied={T|T < 7315424446302937088}]
I20260806 04:58:16.472637 21958 mvcc.cc:205] Tried to move back new op lower bound from 7315424446302937088 to 7315424446131363840. Current Snapshot: MvccSnapshot[applied={T|T < 7315424446302937088 or (T in {7315424446302937088})}]
I20260806 04:58:16.473970 21984 tablet_replica.cc:439] T 781b6631a658446db1b07093b7b6c644 P 8f45f7b26a8f43799df89071dca99b7a: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "8f45f7b26a8f43799df89071dca99b7a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f45f7b26a8f43799df89071dca99b7a" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 39027 } } peers { permanent_uuid: "0c634c0504d446008ab407bb82882031" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 46591 } } peers { permanent_uuid: "ebc08b93691f474698e93a5cc45baa6c" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 36799 } } }
I20260806 04:58:16.474215 21984 tablet_replica.cc:442] T 781b6631a658446db1b07093b7b6c644 P 8f45f7b26a8f43799df89071dca99b7a: This TxnStatusTablet replica's current role is: LEADER
I20260806 04:58:16.474954 21956 mvcc.cc:205] Tried to move back new op lower bound from 7315424446302937088 to 7315424446131363840. Current Snapshot: MvccSnapshot[applied={T|T < 7315424446302937088}]
I20260806 04:58:16.481809 21946 tablet_replica.cc:439] T 781b6631a658446db1b07093b7b6c644 P ebc08b93691f474698e93a5cc45baa6c: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "8f45f7b26a8f43799df89071dca99b7a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f45f7b26a8f43799df89071dca99b7a" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 39027 } } peers { permanent_uuid: "0c634c0504d446008ab407bb82882031" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 46591 } } peers { permanent_uuid: "ebc08b93691f474698e93a5cc45baa6c" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 36799 } } }
I20260806 04:58:16.482057 21946 tablet_replica.cc:442] T 781b6631a658446db1b07093b7b6c644 P ebc08b93691f474698e93a5cc45baa6c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 04:58:16.490257 22010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:58:16.483291 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35814 (local address 127.9.96.65:39027)
0806 04:58:16.484436 (+ 1145us) server_negotiation.cc:206] Beginning negotiation
0806 04:58:16.484442 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 04:58:16.485040 (+ 598us) server_negotiation.cc:407] Connection header received
0806 04:58:16.485243 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:58:16.485247 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:58:16.485334 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:58:16.485425 (+ 91us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 04:58:16.486050 (+ 625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:16.486824 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:16.488286 (+ 1462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:16.488858 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:16.489024 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:58:16.489413 (+ 389us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 04:58:16.489509 (+ 96us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 04:58:16.489625 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0806 04:58:16.489776 (+ 151us) server_negotiation.cc:299] Negotiation successful
0806 04:58:16.489864 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":980}
I20260806 04:58:16.507766 22010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:58:16.500939 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.96.65:50087 (local address 127.9.96.65:39027)
0806 04:58:16.501114 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0806 04:58:16.501121 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 04:58:16.501866 (+ 745us) server_negotiation.cc:407] Connection header received
0806 04:58:16.502064 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:58:16.502068 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:58:16.502129 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:58:16.502196 (+ 67us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 04:58:16.502800 (+ 604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:16.503617 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:16.505781 (+ 2164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:16.506452 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:16.506542 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:58:16.507132 (+ 590us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 04:58:16.507220 (+ 88us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 04:58:16.507291 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0806 04:58:16.507475 (+ 184us) server_negotiation.cc:299] Negotiation successful
0806 04:58:16.507534 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260806 04:58:16.508067 22021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:58:16.501029 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.65:39027 (local address 127.9.96.65:50087)
0806 04:58:16.501753 (+ 724us) negotiation.cc:107] Waiting for socket to connect
0806 04:58:16.501777 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0806 04:58:16.501919 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:58:16.502246 (+ 327us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:58:16.502251 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:58:16.502264 (+ 13us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 04:58:16.502651 (+ 387us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:58:16.502662 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:58:16.503743 (+ 1081us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:58:16.503749 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:58:16.505636 (+ 1887us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:58:16.505652 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:58:16.506711 (+ 1059us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:58:16.506717 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:58:16.506895 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:58:16.506956 (+ 61us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 04:58:16.507331 (+ 375us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 04:58:16.507343 (+ 12us) client_negotiation.cc:769] Sending connection context
0806 04:58:16.507455 (+ 112us) client_negotiation.cc:240] Negotiation successful
0806 04:58:16.507595 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":549,"thread_start_us":383,"threads_started":1}
I20260806 04:58:16.517503 21833 raft_consensus.cc:1261] T cde096dbe74b40deb92026039e59e0eb P ebc08b93691f474698e93a5cc45baa6c [term 1 FOLLOWER]: Refusing update from remote peer 8f45f7b26a8f43799df89071dca99b7a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 04:58:16.517776 21623 raft_consensus.cc:1261] T cde096dbe74b40deb92026039e59e0eb P 0c634c0504d446008ab407bb82882031 [term 1 FOLLOWER]: Refusing update from remote peer 8f45f7b26a8f43799df89071dca99b7a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 04:58:16.518015 21947 consensus_queue.cc:1035] T cde096dbe74b40deb92026039e59e0eb P 8f45f7b26a8f43799df89071dca99b7a [LEADER]: Connected to new peer: Peer: permanent_uuid: "ebc08b93691f474698e93a5cc45baa6c" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 36799 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 04:58:16.518281 21984 consensus_queue.cc:1035] T cde096dbe74b40deb92026039e59e0eb P 8f45f7b26a8f43799df89071dca99b7a [LEADER]: Connected to new peer: Peer: permanent_uuid: "0c634c0504d446008ab407bb82882031" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 46591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 04:58:16.587147 21169 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:57216:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 04:58:16.587508 21169 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:57216:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 04:58:16.590936 21169 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6015b5a9ea39484baf36a8765add32e3: Sending DeleteTablet for 3 replicas of tablet cde096dbe74b40deb92026039e59e0eb
I20260806 04:58:16.592085 21369 tablet_service.cc:1514] Processing DeleteTablet for tablet cde096dbe74b40deb92026039e59e0eb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 04:58:16 UTC) from {username='oryx', principal='oryx/127.9.96.126@KRBTEST.COM'} at 127.0.0.1:35808
I20260806 04:58:16.592101 21802 tablet_service.cc:1514] Processing DeleteTablet for tablet cde096dbe74b40deb92026039e59e0eb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 04:58:16 UTC) from {username='oryx', principal='oryx/127.9.96.126@KRBTEST.COM'} at 127.0.0.1:33152
I20260806 04:58:16.592173 21585 tablet_service.cc:1514] Processing DeleteTablet for tablet cde096dbe74b40deb92026039e59e0eb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 04:58:16 UTC) from {username='oryx', principal='oryx/127.9.96.126@KRBTEST.COM'} at 127.0.0.1:34038
I20260806 04:58:16.603901 22034 tablet_replica.cc:330] T cde096dbe74b40deb92026039e59e0eb P 0c634c0504d446008ab407bb82882031: stopping tablet replica
I20260806 04:58:16.604233 22034 raft_consensus.cc:2229] T cde096dbe74b40deb92026039e59e0eb P 0c634c0504d446008ab407bb82882031 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 04:58:16.604576 22034 raft_consensus.cc:2258] T cde096dbe74b40deb92026039e59e0eb P 0c634c0504d446008ab407bb82882031 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 04:58:16.607882 22030 tablet_replica.cc:330] T cde096dbe74b40deb92026039e59e0eb P ebc08b93691f474698e93a5cc45baa6c: stopping tablet replica
I20260806 04:58:16.608142 22030 raft_consensus.cc:2229] T cde096dbe74b40deb92026039e59e0eb P ebc08b93691f474698e93a5cc45baa6c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 04:58:16.608461 22030 raft_consensus.cc:2258] T cde096dbe74b40deb92026039e59e0eb P ebc08b93691f474698e93a5cc45baa6c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 04:58:16.609711 22030 ts_tablet_manager.cc:1905] T cde096dbe74b40deb92026039e59e0eb P ebc08b93691f474698e93a5cc45baa6c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 04:58:16.612267 22034 ts_tablet_manager.cc:1905] T cde096dbe74b40deb92026039e59e0eb P 0c634c0504d446008ab407bb82882031: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 04:58:16.615300 22033 tablet_replica.cc:330] T cde096dbe74b40deb92026039e59e0eb P 8f45f7b26a8f43799df89071dca99b7a: stopping tablet replica
I20260806 04:58:16.619078 22030 ts_tablet_manager.cc:1918] T cde096dbe74b40deb92026039e59e0eb P ebc08b93691f474698e93a5cc45baa6c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 04:58:16.619230 22030 log.cc:1196] T cde096dbe74b40deb92026039e59e0eb P ebc08b93691f474698e93a5cc45baa6c: Deleting WAL directory at /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785992227050689-9601-0/minicluster-data/ts-2/wal/wals/cde096dbe74b40deb92026039e59e0eb
I20260806 04:58:16.615975 22033 raft_consensus.cc:2229] T cde096dbe74b40deb92026039e59e0eb P 8f45f7b26a8f43799df89071dca99b7a [term 1 LEADER]: Raft consensus shutting down.
I20260806 04:58:16.619761 22030 ts_tablet_manager.cc:1939] T cde096dbe74b40deb92026039e59e0eb P ebc08b93691f474698e93a5cc45baa6c: Deleting consensus metadata
I20260806 04:58:16.620010 22033 raft_consensus.cc:2258] T cde096dbe74b40deb92026039e59e0eb P 8f45f7b26a8f43799df89071dca99b7a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 04:58:16.620222 22034 ts_tablet_manager.cc:1918] T cde096dbe74b40deb92026039e59e0eb P 0c634c0504d446008ab407bb82882031: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 04:58:16.620429 22034 log.cc:1196] T cde096dbe74b40deb92026039e59e0eb P 0c634c0504d446008ab407bb82882031: Deleting WAL directory at /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785992227050689-9601-0/minicluster-data/ts-1/wal/wals/cde096dbe74b40deb92026039e59e0eb
I20260806 04:58:16.620945 22034 ts_tablet_manager.cc:1939] T cde096dbe74b40deb92026039e59e0eb P 0c634c0504d446008ab407bb82882031: Deleting consensus metadata
I20260806 04:58:16.621045 21146 catalog_manager.cc:4841] TS ebc08b93691f474698e93a5cc45baa6c (127.9.96.67:36799): tablet cde096dbe74b40deb92026039e59e0eb (table test-table [id=e654629ac5c246be83f6adb8ee45bf8a]) successfully deleted
I20260806 04:58:16.621815 22033 ts_tablet_manager.cc:1905] T cde096dbe74b40deb92026039e59e0eb P 8f45f7b26a8f43799df89071dca99b7a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 04:58:16.622193 21150 catalog_manager.cc:4841] TS 0c634c0504d446008ab407bb82882031 (127.9.96.66:46591): tablet cde096dbe74b40deb92026039e59e0eb (table test-table [id=e654629ac5c246be83f6adb8ee45bf8a]) successfully deleted
I20260806 04:58:16.628298 22033 ts_tablet_manager.cc:1918] T cde096dbe74b40deb92026039e59e0eb P 8f45f7b26a8f43799df89071dca99b7a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 04:58:16.628474 22033 log.cc:1196] T cde096dbe74b40deb92026039e59e0eb P 8f45f7b26a8f43799df89071dca99b7a: Deleting WAL directory at /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785992227050689-9601-0/minicluster-data/ts-0/wal/wals/cde096dbe74b40deb92026039e59e0eb
I20260806 04:58:16.628926 22033 ts_tablet_manager.cc:1939] T cde096dbe74b40deb92026039e59e0eb P 8f45f7b26a8f43799df89071dca99b7a: Deleting consensus metadata
I20260806 04:58:16.630224 21148 catalog_manager.cc:4841] TS 8f45f7b26a8f43799df89071dca99b7a (127.9.96.65:39027): tablet cde096dbe74b40deb92026039e59e0eb (table test-table [id=e654629ac5c246be83f6adb8ee45bf8a]) successfully deleted
Aug 06 04:58:16 dist-test-slave-x4qh krb5kdc[21106](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992296, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 06 04:58:16 dist-test-slave-x4qh krb5kdc[21106](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.9.96.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 04:58:16 dist-test-slave-x4qh krb5kdc[21106](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.9.96.126@KRBTEST.COM, Server not found in Kerberos database
W20260806 04:58:16.681797 21997 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 04:58:16.669671 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57224 (local address 127.9.96.126:40621)
0806 04:58:16.669941 (+ 270us) server_negotiation.cc:206] Beginning negotiation
0806 04:58:16.669949 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 04:58:16.670294 (+ 345us) server_negotiation.cc:407] Connection header received
0806 04:58:16.670474 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:58:16.670482 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:58:16.670579 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:58:16.670718 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:58:16.672736 (+ 2018us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:16.674049 (+ 1313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:16.675257 (+ 1208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:16.676002 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:16.676144 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:58:16.681523 (+ 5379us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57224: BlockingRecv error: recv got EOF from 127.0.0.1:57224 (error 108)
Metrics: {"server-negotiator.queue_time_us":87}
Aug 06 04:58:16 dist-test-slave-x4qh krb5kdc[21106](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992296, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.9.96.126@KRBTEST.COM
I20260806 04:58:16.699270 21997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:58:16.685026 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57238 (local address 127.9.96.126:40621)
0806 04:58:16.685239 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0806 04:58:16.685246 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 04:58:16.685605 (+ 359us) server_negotiation.cc:407] Connection header received
0806 04:58:16.685816 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:58:16.685822 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:58:16.685893 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:58:16.686001 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:58:16.687111 (+ 1110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:16.688083 (+ 972us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:16.689531 (+ 1448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:16.689944 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:16.690107 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:58:16.693863 (+ 3756us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:58:16.693869 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:58:16.693872 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:58:16.693911 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:58:16.696474 (+ 2563us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:58:16.697108 (+ 634us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:58:16.697115 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:58:16.697118 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:58:16.697247 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:58:16.697653 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:58:16.697659 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:58:16.697662 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:58:16.698002 (+ 340us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:58:16.698281 (+ 279us) server_negotiation.cc:1035] Waiting for connection context
0806 04:58:16.698584 (+ 303us) server_negotiation.cc:299] Negotiation successful
0806 04:58:16.698812 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260806 04:58:16.702847 21169 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57238:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 04:58:16.703507 21169 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 04:58:16.710465 21369 tablet_service.cc:1467] Processing CreateTablet for tablet 7d530ba5df9c478f8ae1a4630a4dca2e (DEFAULT_TABLE table=test-table [id=cae4ee3349d74adbbb26f4cf3a87a526]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 04:58:16.710734 21802 tablet_service.cc:1467] Processing CreateTablet for tablet 7d530ba5df9c478f8ae1a4630a4dca2e (DEFAULT_TABLE table=test-table [id=cae4ee3349d74adbbb26f4cf3a87a526]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 04:58:16.710942 21369 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7d530ba5df9c478f8ae1a4630a4dca2e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 04:58:16.711153 21802 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7d530ba5df9c478f8ae1a4630a4dca2e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 04:58:16.712074 21585 tablet_service.cc:1467] Processing CreateTablet for tablet 7d530ba5df9c478f8ae1a4630a4dca2e (DEFAULT_TABLE table=test-table [id=cae4ee3349d74adbbb26f4cf3a87a526]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 04:58:16.712759 21585 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7d530ba5df9c478f8ae1a4630a4dca2e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 04:58:16.718050 21938 tablet_bootstrap.cc:492] T 7d530ba5df9c478f8ae1a4630a4dca2e P 8f45f7b26a8f43799df89071dca99b7a: Bootstrap starting.
I20260806 04:58:16.718369 21934 tablet_bootstrap.cc:492] T 7d530ba5df9c478f8ae1a4630a4dca2e P ebc08b93691f474698e93a5cc45baa6c: Bootstrap starting.
I20260806 04:58:16.719862 21938 tablet_bootstrap.cc:654] T 7d530ba5df9c478f8ae1a4630a4dca2e P 8f45f7b26a8f43799df89071dca99b7a: Neither blocks nor log segments found. Creating new log.
I20260806 04:58:16.720091 21934 tablet_bootstrap.cc:654] T 7d530ba5df9c478f8ae1a4630a4dca2e P ebc08b93691f474698e93a5cc45baa6c: Neither blocks nor log segments found. Creating new log.
I20260806 04:58:16.720582 21935 tablet_bootstrap.cc:492] T 7d530ba5df9c478f8ae1a4630a4dca2e P 0c634c0504d446008ab407bb82882031: Bootstrap starting.
I20260806 04:58:16.722406 21935 tablet_bootstrap.cc:654] T 7d530ba5df9c478f8ae1a4630a4dca2e P 0c634c0504d446008ab407bb82882031: Neither blocks nor log segments found. Creating new log.
I20260806 04:58:16.730304 21935 tablet_bootstrap.cc:492] T 7d530ba5df9c478f8ae1a4630a4dca2e P 0c634c0504d446008ab407bb82882031: No bootstrap required, opened a new log
I20260806 04:58:16.730471 21938 tablet_bootstrap.cc:492] T 7d530ba5df9c478f8ae1a4630a4dca2e P 8f45f7b26a8f43799df89071dca99b7a: No bootstrap required, opened a new log
I20260806 04:58:16.730453 21935 ts_tablet_manager.cc:1397] T 7d530ba5df9c478f8ae1a4630a4dca2e P 0c634c0504d446008ab407bb82882031: Time spent bootstrapping tablet: real 0.010s user 0.006s sys 0.001s
I20260806 04:58:16.730625 21938 ts_tablet_manager.cc:1397] T 7d530ba5df9c478f8ae1a4630a4dca2e P 8f45f7b26a8f43799df89071dca99b7a: Time spent bootstrapping tablet: real 0.013s user 0.007s sys 0.003s
I20260806 04:58:16.731254 21935 raft_consensus.cc:348] T 7d530ba5df9c478f8ae1a4630a4dca2e P 0c634c0504d446008ab407bb82882031 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f45f7b26a8f43799df89071dca99b7a" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 39027 } } peers { permanent_uuid: "0c634c0504d446008ab407bb82882031" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 46591 } } peers { permanent_uuid: "ebc08b93691f474698e93a5cc45baa6c" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 36799 } }
I20260806 04:58:16.731559 21935 raft_consensus.cc:374] T 7d530ba5df9c478f8ae1a4630a4dca2e P 0c634c0504d446008ab407bb82882031 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 04:58:16.731648 21935 raft_consensus.cc:729] T 7d530ba5df9c478f8ae1a4630a4dca2e P 0c634c0504d446008ab407bb82882031 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0c634c0504d446008ab407bb82882031, State: Initialized, Role: FOLLOWER
I20260806 04:58:16.731511 21938 raft_consensus.cc:348] T 7d530ba5df9c478f8ae1a4630a4dca2e P 8f45f7b26a8f43799df89071dca99b7a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f45f7b26a8f43799df89071dca99b7a" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 39027 } } peers { permanent_uuid: "0c634c0504d446008ab407bb82882031" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 46591 } } peers { permanent_uuid: "ebc08b93691f474698e93a5cc45baa6c" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 36799 } }
I20260806 04:58:16.731812 21938 raft_consensus.cc:374] T 7d530ba5df9c478f8ae1a4630a4dca2e P 8f45f7b26a8f43799df89071dca99b7a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 04:58:16.731937 21938 raft_consensus.cc:729] T 7d530ba5df9c478f8ae1a4630a4dca2e P 8f45f7b26a8f43799df89071dca99b7a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f45f7b26a8f43799df89071dca99b7a, State: Initialized, Role: FOLLOWER
I20260806 04:58:16.732206 21938 consensus_queue.cc:260] T 7d530ba5df9c478f8ae1a4630a4dca2e P 8f45f7b26a8f43799df89071dca99b7a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f45f7b26a8f43799df89071dca99b7a" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 39027 } } peers { permanent_uuid: "0c634c0504d446008ab407bb82882031" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 46591 } } peers { permanent_uuid: "ebc08b93691f474698e93a5cc45baa6c" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 36799 } }
I20260806 04:58:16.732331 21935 consensus_queue.cc:260] T 7d530ba5df9c478f8ae1a4630a4dca2e P 0c634c0504d446008ab407bb82882031 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f45f7b26a8f43799df89071dca99b7a" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 39027 } } peers { permanent_uuid: "0c634c0504d446008ab407bb82882031" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 46591 } } peers { permanent_uuid: "ebc08b93691f474698e93a5cc45baa6c" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 36799 } }
I20260806 04:58:16.733011 21938 ts_tablet_manager.cc:1428] T 7d530ba5df9c478f8ae1a4630a4dca2e P 8f45f7b26a8f43799df89071dca99b7a: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260806 04:58:16.734057 21935 ts_tablet_manager.cc:1428] T 7d530ba5df9c478f8ae1a4630a4dca2e P 0c634c0504d446008ab407bb82882031: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 04:58:16.734266 21934 tablet_bootstrap.cc:492] T 7d530ba5df9c478f8ae1a4630a4dca2e P ebc08b93691f474698e93a5cc45baa6c: No bootstrap required, opened a new log
I20260806 04:58:16.734452 21934 ts_tablet_manager.cc:1397] T 7d530ba5df9c478f8ae1a4630a4dca2e P ebc08b93691f474698e93a5cc45baa6c: Time spent bootstrapping tablet: real 0.016s user 0.010s sys 0.001s
I20260806 04:58:16.735296 21934 raft_consensus.cc:348] T 7d530ba5df9c478f8ae1a4630a4dca2e P ebc08b93691f474698e93a5cc45baa6c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f45f7b26a8f43799df89071dca99b7a" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 39027 } } peers { permanent_uuid: "0c634c0504d446008ab407bb82882031" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 46591 } } peers { permanent_uuid: "ebc08b93691f474698e93a5cc45baa6c" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 36799 } }
I20260806 04:58:16.735599 21934 raft_consensus.cc:374] T 7d530ba5df9c478f8ae1a4630a4dca2e P ebc08b93691f474698e93a5cc45baa6c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 04:58:16.735702 21934 raft_consensus.cc:729] T 7d530ba5df9c478f8ae1a4630a4dca2e P ebc08b93691f474698e93a5cc45baa6c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ebc08b93691f474698e93a5cc45baa6c, State: Initialized, Role: FOLLOWER
I20260806 04:58:16.735957 21934 consensus_queue.cc:260] T 7d530ba5df9c478f8ae1a4630a4dca2e P ebc08b93691f474698e93a5cc45baa6c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f45f7b26a8f43799df89071dca99b7a" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 39027 } } peers { permanent_uuid: "0c634c0504d446008ab407bb82882031" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 46591 } } peers { permanent_uuid: "ebc08b93691f474698e93a5cc45baa6c" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 36799 } }
I20260806 04:58:16.737105 21934 ts_tablet_manager.cc:1428] T 7d530ba5df9c478f8ae1a4630a4dca2e P ebc08b93691f474698e93a5cc45baa6c: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260806 04:58:16.805877 21984 raft_consensus.cc:482] T 7d530ba5df9c478f8ae1a4630a4dca2e P 8f45f7b26a8f43799df89071dca99b7a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 04:58:16.806092 21984 raft_consensus.cc:504] T 7d530ba5df9c478f8ae1a4630a4dca2e P 8f45f7b26a8f43799df89071dca99b7a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f45f7b26a8f43799df89071dca99b7a" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 39027 } } peers { permanent_uuid: "0c634c0504d446008ab407bb82882031" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 46591 } } peers { permanent_uuid: "ebc08b93691f474698e93a5cc45baa6c" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 36799 } }
I20260806 04:58:16.806834 21984 leader_election.cc:288] T 7d530ba5df9c478f8ae1a4630a4dca2e P 8f45f7b26a8f43799df89071dca99b7a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0c634c0504d446008ab407bb82882031 (127.9.96.66:46591), ebc08b93691f474698e93a5cc45baa6c (127.9.96.67:36799)
I20260806 04:58:16.807536 21623 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7d530ba5df9c478f8ae1a4630a4dca2e" candidate_uuid: "8f45f7b26a8f43799df89071dca99b7a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0c634c0504d446008ab407bb82882031" is_pre_election: true
I20260806 04:58:16.807765 21833 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7d530ba5df9c478f8ae1a4630a4dca2e" candidate_uuid: "8f45f7b26a8f43799df89071dca99b7a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ebc08b93691f474698e93a5cc45baa6c" is_pre_election: true
I20260806 04:58:16.807826 21623 raft_consensus.cc:2454] T 7d530ba5df9c478f8ae1a4630a4dca2e P 0c634c0504d446008ab407bb82882031 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f45f7b26a8f43799df89071dca99b7a in term 0.
I20260806 04:58:16.808000 21833 raft_consensus.cc:2454] T 7d530ba5df9c478f8ae1a4630a4dca2e P ebc08b93691f474698e93a5cc45baa6c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f45f7b26a8f43799df89071dca99b7a in term 0.
I20260806 04:58:16.808461 21293 leader_election.cc:302] T 7d530ba5df9c478f8ae1a4630a4dca2e P 8f45f7b26a8f43799df89071dca99b7a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8f45f7b26a8f43799df89071dca99b7a, ebc08b93691f474698e93a5cc45baa6c; no voters:
I20260806 04:58:16.808844 21984 raft_consensus.cc:2781] T 7d530ba5df9c478f8ae1a4630a4dca2e P 8f45f7b26a8f43799df89071dca99b7a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 04:58:16.809010 21984 raft_consensus.cc:482] T 7d530ba5df9c478f8ae1a4630a4dca2e P 8f45f7b26a8f43799df89071dca99b7a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 04:58:16.809131 21984 raft_consensus.cc:3037] T 7d530ba5df9c478f8ae1a4630a4dca2e P 8f45f7b26a8f43799df89071dca99b7a [term 0 FOLLOWER]: Advancing to term 1
I20260806 04:58:16.812776 21984 raft_consensus.cc:504] T 7d530ba5df9c478f8ae1a4630a4dca2e P 8f45f7b26a8f43799df89071dca99b7a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f45f7b26a8f43799df89071dca99b7a" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 39027 } } peers { permanent_uuid: "0c634c0504d446008ab407bb82882031" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 46591 } } peers { permanent_uuid: "ebc08b93691f474698e93a5cc45baa6c" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 36799 } }
I20260806 04:58:16.813548 21984 leader_election.cc:288] T 7d530ba5df9c478f8ae1a4630a4dca2e P 8f45f7b26a8f43799df89071dca99b7a [CANDIDATE]: Term 1 election: Requested vote from peers 0c634c0504d446008ab407bb82882031 (127.9.96.66:46591), ebc08b93691f474698e93a5cc45baa6c (127.9.96.67:36799)
I20260806 04:58:16.814031 21833 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7d530ba5df9c478f8ae1a4630a4dca2e" candidate_uuid: "8f45f7b26a8f43799df89071dca99b7a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ebc08b93691f474698e93a5cc45baa6c"
I20260806 04:58:16.814056 21623 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7d530ba5df9c478f8ae1a4630a4dca2e" candidate_uuid: "8f45f7b26a8f43799df89071dca99b7a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0c634c0504d446008ab407bb82882031"
I20260806 04:58:16.814249 21833 raft_consensus.cc:3037] T 7d530ba5df9c478f8ae1a4630a4dca2e P ebc08b93691f474698e93a5cc45baa6c [term 0 FOLLOWER]: Advancing to term 1
I20260806 04:58:16.814275 21623 raft_consensus.cc:3037] T 7d530ba5df9c478f8ae1a4630a4dca2e P 0c634c0504d446008ab407bb82882031 [term 0 FOLLOWER]: Advancing to term 1
I20260806 04:58:16.817068 21833 raft_consensus.cc:2454] T 7d530ba5df9c478f8ae1a4630a4dca2e P ebc08b93691f474698e93a5cc45baa6c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f45f7b26a8f43799df89071dca99b7a in term 1.
I20260806 04:58:16.817559 21293 leader_election.cc:302] T 7d530ba5df9c478f8ae1a4630a4dca2e P 8f45f7b26a8f43799df89071dca99b7a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8f45f7b26a8f43799df89071dca99b7a, ebc08b93691f474698e93a5cc45baa6c; no voters:
I20260806 04:58:16.817721 21623 raft_consensus.cc:2454] T 7d530ba5df9c478f8ae1a4630a4dca2e P 0c634c0504d446008ab407bb82882031 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f45f7b26a8f43799df89071dca99b7a in term 1.
I20260806 04:58:16.817906 21984 raft_consensus.cc:2781] T 7d530ba5df9c478f8ae1a4630a4dca2e P 8f45f7b26a8f43799df89071dca99b7a [term 1 FOLLOWER]: Leader election won for term 1
I20260806 04:58:16.818078 21984 raft_consensus.cc:686] T 7d530ba5df9c478f8ae1a4630a4dca2e P 8f45f7b26a8f43799df89071dca99b7a [term 1 LEADER]: Becoming Leader. State: Replica: 8f45f7b26a8f43799df89071dca99b7a, State: Running, Role: LEADER
I20260806 04:58:16.818377 21984 consensus_queue.cc:237] T 7d530ba5df9c478f8ae1a4630a4dca2e P 8f45f7b26a8f43799df89071dca99b7a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f45f7b26a8f43799df89071dca99b7a" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 39027 } } peers { permanent_uuid: "0c634c0504d446008ab407bb82882031" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 46591 } } peers { permanent_uuid: "ebc08b93691f474698e93a5cc45baa6c" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 36799 } }
I20260806 04:58:16.821547 21169 catalog_manager.cc:5495] T 7d530ba5df9c478f8ae1a4630a4dca2e P 8f45f7b26a8f43799df89071dca99b7a reported cstate change: term changed from 0 to 1, leader changed from <none> to 8f45f7b26a8f43799df89071dca99b7a (127.9.96.65). New cstate: current_term: 1 leader_uuid: "8f45f7b26a8f43799df89071dca99b7a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f45f7b26a8f43799df89071dca99b7a" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 39027 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0c634c0504d446008ab407bb82882031" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 46591 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ebc08b93691f474698e93a5cc45baa6c" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 36799 } health_report { overall_health: UNKNOWN } } }
I20260806 04:58:16.841452 22010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:58:16.835498 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35822 (local address 127.9.96.65:39027)
0806 04:58:16.835698 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0806 04:58:16.835720 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0806 04:58:16.835811 (+ 91us) server_negotiation.cc:407] Connection header received
0806 04:58:16.835989 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:58:16.835995 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:58:16.836092 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:58:16.836223 (+ 131us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 04:58:16.836585 (+ 362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:16.837853 (+ 1268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:16.839205 (+ 1352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:16.840045 (+ 840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:16.840210 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:58:16.840591 (+ 381us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 04:58:16.840791 (+ 200us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 04:58:16.840911 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0806 04:58:16.841014 (+ 103us) server_negotiation.cc:299] Negotiation successful
0806 04:58:16.841108 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260806 04:58:16.844466 21833 raft_consensus.cc:1261] T 7d530ba5df9c478f8ae1a4630a4dca2e P ebc08b93691f474698e93a5cc45baa6c [term 1 FOLLOWER]: Refusing update from remote peer 8f45f7b26a8f43799df89071dca99b7a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 04:58:16.844810 21623 raft_consensus.cc:1261] T 7d530ba5df9c478f8ae1a4630a4dca2e P 0c634c0504d446008ab407bb82882031 [term 1 FOLLOWER]: Refusing update from remote peer 8f45f7b26a8f43799df89071dca99b7a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 04:58:16.845144 22029 consensus_queue.cc:1035] T 7d530ba5df9c478f8ae1a4630a4dca2e P 8f45f7b26a8f43799df89071dca99b7a [LEADER]: Connected to new peer: Peer: permanent_uuid: "ebc08b93691f474698e93a5cc45baa6c" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 36799 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 04:58:16.846451 22029 consensus_queue.cc:1035] T 7d530ba5df9c478f8ae1a4630a4dca2e P 8f45f7b26a8f43799df89071dca99b7a [LEADER]: Connected to new peer: Peer: permanent_uuid: "0c634c0504d446008ab407bb82882031" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 46591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 04:58:16.863719 21166 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57238:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 04:58:16.864042 21166 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57238:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 04:58:16.868630 21166 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6015b5a9ea39484baf36a8765add32e3: Sending DeleteTablet for 3 replicas of tablet 7d530ba5df9c478f8ae1a4630a4dca2e
I20260806 04:58:16.869585 21369 tablet_service.cc:1514] Processing DeleteTablet for tablet 7d530ba5df9c478f8ae1a4630a4dca2e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 04:58:16 UTC) from {username='oryx', principal='oryx/127.9.96.126@KRBTEST.COM'} at 127.0.0.1:35808
I20260806 04:58:16.869614 21585 tablet_service.cc:1514] Processing DeleteTablet for tablet 7d530ba5df9c478f8ae1a4630a4dca2e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 04:58:16 UTC) from {username='oryx', principal='oryx/127.9.96.126@KRBTEST.COM'} at 127.0.0.1:34038
I20260806 04:58:16.869895 22033 tablet_replica.cc:330] T 7d530ba5df9c478f8ae1a4630a4dca2e P 8f45f7b26a8f43799df89071dca99b7a: stopping tablet replica
I20260806 04:58:16.870057 22033 raft_consensus.cc:2229] T 7d530ba5df9c478f8ae1a4630a4dca2e P 8f45f7b26a8f43799df89071dca99b7a [term 1 LEADER]: Raft consensus shutting down.
I20260806 04:58:16.870334 22034 tablet_replica.cc:330] T 7d530ba5df9c478f8ae1a4630a4dca2e P 0c634c0504d446008ab407bb82882031: stopping tablet replica
I20260806 04:58:16.870363 22033 raft_consensus.cc:2258] T 7d530ba5df9c478f8ae1a4630a4dca2e P 8f45f7b26a8f43799df89071dca99b7a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 04:58:16.870548 22034 raft_consensus.cc:2229] T 7d530ba5df9c478f8ae1a4630a4dca2e P 0c634c0504d446008ab407bb82882031 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 04:58:16.870820 22034 raft_consensus.cc:2258] T 7d530ba5df9c478f8ae1a4630a4dca2e P 0c634c0504d446008ab407bb82882031 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 04:58:16.872432 22033 ts_tablet_manager.cc:1905] T 7d530ba5df9c478f8ae1a4630a4dca2e P 8f45f7b26a8f43799df89071dca99b7a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 04:58:16.872742 22034 ts_tablet_manager.cc:1905] T 7d530ba5df9c478f8ae1a4630a4dca2e P 0c634c0504d446008ab407bb82882031: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 04:58:16.875990 21369 tablet_service.cc:1514] Processing DeleteTablet for tablet 7d530ba5df9c478f8ae1a4630a4dca2e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 04:58:16 UTC) from {username='oryx', principal='oryx/127.9.96.126@KRBTEST.COM'} at 127.0.0.1:35808
I20260806 04:58:16.876746 21585 tablet_service.cc:1514] Processing DeleteTablet for tablet 7d530ba5df9c478f8ae1a4630a4dca2e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 04:58:16 UTC) from {username='oryx', principal='oryx/127.9.96.126@KRBTEST.COM'} at 127.0.0.1:34038
I20260806 04:58:16.877489 21802 tablet_service.cc:1514] Processing DeleteTablet for tablet 7d530ba5df9c478f8ae1a4630a4dca2e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 04:58:16 UTC) from {username='oryx', principal='oryx/127.9.96.126@KRBTEST.COM'} at 127.0.0.1:33152
I20260806 04:58:16.877760 22030 tablet_replica.cc:330] T 7d530ba5df9c478f8ae1a4630a4dca2e P ebc08b93691f474698e93a5cc45baa6c: stopping tablet replica
I20260806 04:58:16.877986 22030 raft_consensus.cc:2229] T 7d530ba5df9c478f8ae1a4630a4dca2e P ebc08b93691f474698e93a5cc45baa6c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 04:58:16.878216 22030 raft_consensus.cc:2258] T 7d530ba5df9c478f8ae1a4630a4dca2e P ebc08b93691f474698e93a5cc45baa6c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 04:58:16.879333 22034 ts_tablet_manager.cc:1918] T 7d530ba5df9c478f8ae1a4630a4dca2e P 0c634c0504d446008ab407bb82882031: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 04:58:16.879513 22034 log.cc:1196] T 7d530ba5df9c478f8ae1a4630a4dca2e P 0c634c0504d446008ab407bb82882031: Deleting WAL directory at /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785992227050689-9601-0/minicluster-data/ts-1/wal/wals/7d530ba5df9c478f8ae1a4630a4dca2e
I20260806 04:58:16.880009 22034 ts_tablet_manager.cc:1939] T 7d530ba5df9c478f8ae1a4630a4dca2e P 0c634c0504d446008ab407bb82882031: Deleting consensus metadata
I20260806 04:58:16.881033 21150 catalog_manager.cc:4841] TS 0c634c0504d446008ab407bb82882031 (127.9.96.66:46591): tablet 7d530ba5df9c478f8ae1a4630a4dca2e (table test-table [id=cae4ee3349d74adbbb26f4cf3a87a526]) successfully deleted
W20260806 04:58:16.881593 21150 catalog_manager.cc:4808] TS 0c634c0504d446008ab407bb82882031 (127.9.96.66:46591): delete failed for tablet 7d530ba5df9c478f8ae1a4630a4dca2e because the tablet was not found. No further retry: Not found: Tablet not found: 7d530ba5df9c478f8ae1a4630a4dca2e
I20260806 04:58:16.883994 22033 ts_tablet_manager.cc:1918] T 7d530ba5df9c478f8ae1a4630a4dca2e P 8f45f7b26a8f43799df89071dca99b7a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 04:58:16.884203 22033 log.cc:1196] T 7d530ba5df9c478f8ae1a4630a4dca2e P 8f45f7b26a8f43799df89071dca99b7a: Deleting WAL directory at /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785992227050689-9601-0/minicluster-data/ts-0/wal/wals/7d530ba5df9c478f8ae1a4630a4dca2e
I20260806 04:58:16.884656 22033 ts_tablet_manager.cc:1939] T 7d530ba5df9c478f8ae1a4630a4dca2e P 8f45f7b26a8f43799df89071dca99b7a: Deleting consensus metadata
I20260806 04:58:16.885654 21148 catalog_manager.cc:4841] TS 8f45f7b26a8f43799df89071dca99b7a (127.9.96.65:39027): tablet 7d530ba5df9c478f8ae1a4630a4dca2e (table test-table [id=cae4ee3349d74adbbb26f4cf3a87a526]) successfully deleted
W20260806 04:58:16.885926 21148 catalog_manager.cc:4808] TS 8f45f7b26a8f43799df89071dca99b7a (127.9.96.65:39027): delete failed for tablet 7d530ba5df9c478f8ae1a4630a4dca2e because the tablet was not found. No further retry: Not found: Tablet not found: 7d530ba5df9c478f8ae1a4630a4dca2e
I20260806 04:58:16.886118 21166 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57238:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 04:58:16.886751 21166 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 04:58:16.888504 22030 ts_tablet_manager.cc:1905] T 7d530ba5df9c478f8ae1a4630a4dca2e P ebc08b93691f474698e93a5cc45baa6c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 04:58:16.894374 22030 ts_tablet_manager.cc:1918] T 7d530ba5df9c478f8ae1a4630a4dca2e P ebc08b93691f474698e93a5cc45baa6c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 04:58:16.894562 22030 log.cc:1196] T 7d530ba5df9c478f8ae1a4630a4dca2e P ebc08b93691f474698e93a5cc45baa6c: Deleting WAL directory at /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785992227050689-9601-0/minicluster-data/ts-2/wal/wals/7d530ba5df9c478f8ae1a4630a4dca2e
I20260806 04:58:16.897265 22030 ts_tablet_manager.cc:1939] T 7d530ba5df9c478f8ae1a4630a4dca2e P ebc08b93691f474698e93a5cc45baa6c: Deleting consensus metadata
I20260806 04:58:16.899569 21146 catalog_manager.cc:4841] TS ebc08b93691f474698e93a5cc45baa6c (127.9.96.67:36799): tablet 7d530ba5df9c478f8ae1a4630a4dca2e (table test-table [id=cae4ee3349d74adbbb26f4cf3a87a526]) successfully deleted
I20260806 04:58:16.900094 21802 tablet_service.cc:1467] Processing CreateTablet for tablet 3192cb5fc49d40f8a940e03da41b405d (DEFAULT_TABLE table=test-table [id=8962763aa3af43e396ca57444cb0ef0b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 04:58:16.900512 21802 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3192cb5fc49d40f8a940e03da41b405d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 04:58:16.900889 21369 tablet_service.cc:1467] Processing CreateTablet for tablet 3192cb5fc49d40f8a940e03da41b405d (DEFAULT_TABLE table=test-table [id=8962763aa3af43e396ca57444cb0ef0b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 04:58:16.901338 21369 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3192cb5fc49d40f8a940e03da41b405d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 04:58:16.906767 21934 tablet_bootstrap.cc:492] T 3192cb5fc49d40f8a940e03da41b405d P ebc08b93691f474698e93a5cc45baa6c: Bootstrap starting.
I20260806 04:58:16.907883 21585 tablet_service.cc:1467] Processing CreateTablet for tablet 3192cb5fc49d40f8a940e03da41b405d (DEFAULT_TABLE table=test-table [id=8962763aa3af43e396ca57444cb0ef0b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 04:58:16.908367 21585 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3192cb5fc49d40f8a940e03da41b405d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 04:58:16.908373 21934 tablet_bootstrap.cc:654] T 3192cb5fc49d40f8a940e03da41b405d P ebc08b93691f474698e93a5cc45baa6c: Neither blocks nor log segments found. Creating new log.
I20260806 04:58:16.916707 21938 tablet_bootstrap.cc:492] T 3192cb5fc49d40f8a940e03da41b405d P 8f45f7b26a8f43799df89071dca99b7a: Bootstrap starting.
I20260806 04:58:16.918378 21935 tablet_bootstrap.cc:492] T 3192cb5fc49d40f8a940e03da41b405d P 0c634c0504d446008ab407bb82882031: Bootstrap starting.
I20260806 04:58:16.918887 21938 tablet_bootstrap.cc:654] T 3192cb5fc49d40f8a940e03da41b405d P 8f45f7b26a8f43799df89071dca99b7a: Neither blocks nor log segments found. Creating new log.
I20260806 04:58:16.920197 21935 tablet_bootstrap.cc:654] T 3192cb5fc49d40f8a940e03da41b405d P 0c634c0504d446008ab407bb82882031: Neither blocks nor log segments found. Creating new log.
I20260806 04:58:16.921445 21934 tablet_bootstrap.cc:492] T 3192cb5fc49d40f8a940e03da41b405d P ebc08b93691f474698e93a5cc45baa6c: No bootstrap required, opened a new log
I20260806 04:58:16.921656 21934 ts_tablet_manager.cc:1397] T 3192cb5fc49d40f8a940e03da41b405d P ebc08b93691f474698e93a5cc45baa6c: Time spent bootstrapping tablet: real 0.015s user 0.007s sys 0.004s
I20260806 04:58:16.922457 21934 raft_consensus.cc:348] T 3192cb5fc49d40f8a940e03da41b405d P ebc08b93691f474698e93a5cc45baa6c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f45f7b26a8f43799df89071dca99b7a" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 39027 } } peers { permanent_uuid: "0c634c0504d446008ab407bb82882031" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 46591 } } peers { permanent_uuid: "ebc08b93691f474698e93a5cc45baa6c" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 36799 } }
I20260806 04:58:16.922776 21934 raft_consensus.cc:374] T 3192cb5fc49d40f8a940e03da41b405d P ebc08b93691f474698e93a5cc45baa6c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 04:58:16.922915 21934 raft_consensus.cc:729] T 3192cb5fc49d40f8a940e03da41b405d P ebc08b93691f474698e93a5cc45baa6c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ebc08b93691f474698e93a5cc45baa6c, State: Initialized, Role: FOLLOWER
I20260806 04:58:16.924273 21934 consensus_queue.cc:260] T 3192cb5fc49d40f8a940e03da41b405d P ebc08b93691f474698e93a5cc45baa6c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f45f7b26a8f43799df89071dca99b7a" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 39027 } } peers { permanent_uuid: "0c634c0504d446008ab407bb82882031" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 46591 } } peers { permanent_uuid: "ebc08b93691f474698e93a5cc45baa6c" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 36799 } }
I20260806 04:58:16.924932 21934 ts_tablet_manager.cc:1428] T 3192cb5fc49d40f8a940e03da41b405d P ebc08b93691f474698e93a5cc45baa6c: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20260806 04:58:16.926568 21935 tablet_bootstrap.cc:492] T 3192cb5fc49d40f8a940e03da41b405d P 0c634c0504d446008ab407bb82882031: No bootstrap required, opened a new log
I20260806 04:58:16.926730 21935 ts_tablet_manager.cc:1397] T 3192cb5fc49d40f8a940e03da41b405d P 0c634c0504d446008ab407bb82882031: Time spent bootstrapping tablet: real 0.008s user 0.004s sys 0.003s
I20260806 04:58:16.927552 21935 raft_consensus.cc:348] T 3192cb5fc49d40f8a940e03da41b405d P 0c634c0504d446008ab407bb82882031 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f45f7b26a8f43799df89071dca99b7a" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 39027 } } peers { permanent_uuid: "0c634c0504d446008ab407bb82882031" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 46591 } } peers { permanent_uuid: "ebc08b93691f474698e93a5cc45baa6c" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 36799 } }
I20260806 04:58:16.927793 21935 raft_consensus.cc:374] T 3192cb5fc49d40f8a940e03da41b405d P 0c634c0504d446008ab407bb82882031 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 04:58:16.927896 21935 raft_consensus.cc:729] T 3192cb5fc49d40f8a940e03da41b405d P 0c634c0504d446008ab407bb82882031 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0c634c0504d446008ab407bb82882031, State: Initialized, Role: FOLLOWER
I20260806 04:58:16.928206 21935 consensus_queue.cc:260] T 3192cb5fc49d40f8a940e03da41b405d P 0c634c0504d446008ab407bb82882031 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f45f7b26a8f43799df89071dca99b7a" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 39027 } } peers { permanent_uuid: "0c634c0504d446008ab407bb82882031" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 46591 } } peers { permanent_uuid: "ebc08b93691f474698e93a5cc45baa6c" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 36799 } }
I20260806 04:58:16.928622 21938 tablet_bootstrap.cc:492] T 3192cb5fc49d40f8a940e03da41b405d P 8f45f7b26a8f43799df89071dca99b7a: No bootstrap required, opened a new log
I20260806 04:58:16.928825 21935 ts_tablet_manager.cc:1428] T 3192cb5fc49d40f8a940e03da41b405d P 0c634c0504d446008ab407bb82882031: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260806 04:58:16.928874 21938 ts_tablet_manager.cc:1397] T 3192cb5fc49d40f8a940e03da41b405d P 8f45f7b26a8f43799df89071dca99b7a: Time spent bootstrapping tablet: real 0.012s user 0.008s sys 0.000s
I20260806 04:58:16.929731 21938 raft_consensus.cc:348] T 3192cb5fc49d40f8a940e03da41b405d P 8f45f7b26a8f43799df89071dca99b7a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f45f7b26a8f43799df89071dca99b7a" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 39027 } } peers { permanent_uuid: "0c634c0504d446008ab407bb82882031" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 46591 } } peers { permanent_uuid: "ebc08b93691f474698e93a5cc45baa6c" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 36799 } }
I20260806 04:58:16.930074 21938 raft_consensus.cc:374] T 3192cb5fc49d40f8a940e03da41b405d P 8f45f7b26a8f43799df89071dca99b7a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 04:58:16.930162 21938 raft_consensus.cc:729] T 3192cb5fc49d40f8a940e03da41b405d P 8f45f7b26a8f43799df89071dca99b7a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f45f7b26a8f43799df89071dca99b7a, State: Initialized, Role: FOLLOWER
I20260806 04:58:16.930409 21938 consensus_queue.cc:260] T 3192cb5fc49d40f8a940e03da41b405d P 8f45f7b26a8f43799df89071dca99b7a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f45f7b26a8f43799df89071dca99b7a" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 39027 } } peers { permanent_uuid: "0c634c0504d446008ab407bb82882031" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 46591 } } peers { permanent_uuid: "ebc08b93691f474698e93a5cc45baa6c" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 36799 } }
I20260806 04:58:16.931463 21938 ts_tablet_manager.cc:1428] T 3192cb5fc49d40f8a940e03da41b405d P 8f45f7b26a8f43799df89071dca99b7a: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260806 04:58:17.164841 22029 raft_consensus.cc:482] T 3192cb5fc49d40f8a940e03da41b405d P 8f45f7b26a8f43799df89071dca99b7a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 04:58:17.165091 22029 raft_consensus.cc:504] T 3192cb5fc49d40f8a940e03da41b405d P 8f45f7b26a8f43799df89071dca99b7a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f45f7b26a8f43799df89071dca99b7a" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 39027 } } peers { permanent_uuid: "0c634c0504d446008ab407bb82882031" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 46591 } } peers { permanent_uuid: "ebc08b93691f474698e93a5cc45baa6c" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 36799 } }
I20260806 04:58:17.166013 22029 leader_election.cc:288] T 3192cb5fc49d40f8a940e03da41b405d P 8f45f7b26a8f43799df89071dca99b7a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0c634c0504d446008ab407bb82882031 (127.9.96.66:46591), ebc08b93691f474698e93a5cc45baa6c (127.9.96.67:36799)
I20260806 04:58:17.166568 21622 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3192cb5fc49d40f8a940e03da41b405d" candidate_uuid: "8f45f7b26a8f43799df89071dca99b7a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0c634c0504d446008ab407bb82882031" is_pre_election: true
I20260806 04:58:17.166636 21833 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3192cb5fc49d40f8a940e03da41b405d" candidate_uuid: "8f45f7b26a8f43799df89071dca99b7a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ebc08b93691f474698e93a5cc45baa6c" is_pre_election: true
I20260806 04:58:17.166839 21622 raft_consensus.cc:2454] T 3192cb5fc49d40f8a940e03da41b405d P 0c634c0504d446008ab407bb82882031 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f45f7b26a8f43799df89071dca99b7a in term 0.
I20260806 04:58:17.166901 21833 raft_consensus.cc:2454] T 3192cb5fc49d40f8a940e03da41b405d P ebc08b93691f474698e93a5cc45baa6c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f45f7b26a8f43799df89071dca99b7a in term 0.
I20260806 04:58:17.167344 21298 leader_election.cc:302] T 3192cb5fc49d40f8a940e03da41b405d P 8f45f7b26a8f43799df89071dca99b7a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0c634c0504d446008ab407bb82882031, 8f45f7b26a8f43799df89071dca99b7a; no voters:
I20260806 04:58:17.167654 22029 raft_consensus.cc:2781] T 3192cb5fc49d40f8a940e03da41b405d P 8f45f7b26a8f43799df89071dca99b7a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 04:58:17.167798 22029 raft_consensus.cc:482] T 3192cb5fc49d40f8a940e03da41b405d P 8f45f7b26a8f43799df89071dca99b7a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 04:58:17.167923 22029 raft_consensus.cc:3037] T 3192cb5fc49d40f8a940e03da41b405d P 8f45f7b26a8f43799df89071dca99b7a [term 0 FOLLOWER]: Advancing to term 1
I20260806 04:58:17.171253 22029 raft_consensus.cc:504] T 3192cb5fc49d40f8a940e03da41b405d P 8f45f7b26a8f43799df89071dca99b7a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f45f7b26a8f43799df89071dca99b7a" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 39027 } } peers { permanent_uuid: "0c634c0504d446008ab407bb82882031" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 46591 } } peers { permanent_uuid: "ebc08b93691f474698e93a5cc45baa6c" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 36799 } }
I20260806 04:58:17.171954 22029 leader_election.cc:288] T 3192cb5fc49d40f8a940e03da41b405d P 8f45f7b26a8f43799df89071dca99b7a [CANDIDATE]: Term 1 election: Requested vote from peers 0c634c0504d446008ab407bb82882031 (127.9.96.66:46591), ebc08b93691f474698e93a5cc45baa6c (127.9.96.67:36799)
I20260806 04:58:17.172465 21622 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3192cb5fc49d40f8a940e03da41b405d" candidate_uuid: "8f45f7b26a8f43799df89071dca99b7a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0c634c0504d446008ab407bb82882031"
I20260806 04:58:17.172554 21833 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3192cb5fc49d40f8a940e03da41b405d" candidate_uuid: "8f45f7b26a8f43799df89071dca99b7a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ebc08b93691f474698e93a5cc45baa6c"
I20260806 04:58:17.172732 21622 raft_consensus.cc:3037] T 3192cb5fc49d40f8a940e03da41b405d P 0c634c0504d446008ab407bb82882031 [term 0 FOLLOWER]: Advancing to term 1
I20260806 04:58:17.172762 21833 raft_consensus.cc:3037] T 3192cb5fc49d40f8a940e03da41b405d P ebc08b93691f474698e93a5cc45baa6c [term 0 FOLLOWER]: Advancing to term 1
I20260806 04:58:17.175927 21622 raft_consensus.cc:2454] T 3192cb5fc49d40f8a940e03da41b405d P 0c634c0504d446008ab407bb82882031 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f45f7b26a8f43799df89071dca99b7a in term 1.
I20260806 04:58:17.175957 21833 raft_consensus.cc:2454] T 3192cb5fc49d40f8a940e03da41b405d P ebc08b93691f474698e93a5cc45baa6c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f45f7b26a8f43799df89071dca99b7a in term 1.
I20260806 04:58:17.176467 21293 leader_election.cc:302] T 3192cb5fc49d40f8a940e03da41b405d P 8f45f7b26a8f43799df89071dca99b7a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8f45f7b26a8f43799df89071dca99b7a, ebc08b93691f474698e93a5cc45baa6c; no voters:
I20260806 04:58:17.176828 22029 raft_consensus.cc:2781] T 3192cb5fc49d40f8a940e03da41b405d P 8f45f7b26a8f43799df89071dca99b7a [term 1 FOLLOWER]: Leader election won for term 1
I20260806 04:58:17.177088 22029 raft_consensus.cc:686] T 3192cb5fc49d40f8a940e03da41b405d P 8f45f7b26a8f43799df89071dca99b7a [term 1 LEADER]: Becoming Leader. State: Replica: 8f45f7b26a8f43799df89071dca99b7a, State: Running, Role: LEADER
I20260806 04:58:17.177407 22029 consensus_queue.cc:237] T 3192cb5fc49d40f8a940e03da41b405d P 8f45f7b26a8f43799df89071dca99b7a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f45f7b26a8f43799df89071dca99b7a" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 39027 } } peers { permanent_uuid: "0c634c0504d446008ab407bb82882031" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 46591 } } peers { permanent_uuid: "ebc08b93691f474698e93a5cc45baa6c" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 36799 } }
I20260806 04:58:17.180783 21166 catalog_manager.cc:5495] T 3192cb5fc49d40f8a940e03da41b405d P 8f45f7b26a8f43799df89071dca99b7a reported cstate change: term changed from 0 to 1, leader changed from <none> to 8f45f7b26a8f43799df89071dca99b7a (127.9.96.65). New cstate: current_term: 1 leader_uuid: "8f45f7b26a8f43799df89071dca99b7a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f45f7b26a8f43799df89071dca99b7a" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 39027 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0c634c0504d446008ab407bb82882031" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 46591 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ebc08b93691f474698e93a5cc45baa6c" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 36799 } health_report { overall_health: UNKNOWN } } }
I20260806 04:58:17.232100 21833 raft_consensus.cc:1261] T 3192cb5fc49d40f8a940e03da41b405d P ebc08b93691f474698e93a5cc45baa6c [term 1 FOLLOWER]: Refusing update from remote peer 8f45f7b26a8f43799df89071dca99b7a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 04:58:17.232359 21622 raft_consensus.cc:1261] T 3192cb5fc49d40f8a940e03da41b405d P 0c634c0504d446008ab407bb82882031 [term 1 FOLLOWER]: Refusing update from remote peer 8f45f7b26a8f43799df89071dca99b7a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 04:58:17.232635 22029 consensus_queue.cc:1035] T 3192cb5fc49d40f8a940e03da41b405d P 8f45f7b26a8f43799df89071dca99b7a [LEADER]: Connected to new peer: Peer: permanent_uuid: "ebc08b93691f474698e93a5cc45baa6c" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 36799 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 04:58:17.233065 21947 consensus_queue.cc:1035] T 3192cb5fc49d40f8a940e03da41b405d P 8f45f7b26a8f43799df89071dca99b7a [LEADER]: Connected to new peer: Peer: permanent_uuid: "0c634c0504d446008ab407bb82882031" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 46591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 04:58:17.302309 21166 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57238:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 04:58:17.302595 21166 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57238:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 04:58:17.305900 21166 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6015b5a9ea39484baf36a8765add32e3: Sending DeleteTablet for 3 replicas of tablet 3192cb5fc49d40f8a940e03da41b405d
I20260806 04:58:17.306855 21585 tablet_service.cc:1514] Processing DeleteTablet for tablet 3192cb5fc49d40f8a940e03da41b405d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 04:58:17 UTC) from {username='oryx', principal='oryx/127.9.96.126@KRBTEST.COM'} at 127.0.0.1:34038
I20260806 04:58:17.307193 22034 tablet_replica.cc:330] T 3192cb5fc49d40f8a940e03da41b405d P 0c634c0504d446008ab407bb82882031: stopping tablet replica
I20260806 04:58:17.307301 21802 tablet_service.cc:1514] Processing DeleteTablet for tablet 3192cb5fc49d40f8a940e03da41b405d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 04:58:17 UTC) from {username='oryx', principal='oryx/127.9.96.126@KRBTEST.COM'} at 127.0.0.1:33152
I20260806 04:58:17.307333 21369 tablet_service.cc:1514] Processing DeleteTablet for tablet 3192cb5fc49d40f8a940e03da41b405d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 04:58:17 UTC) from {username='oryx', principal='oryx/127.9.96.126@KRBTEST.COM'} at 127.0.0.1:35808
I20260806 04:58:17.307426 22034 raft_consensus.cc:2229] T 3192cb5fc49d40f8a940e03da41b405d P 0c634c0504d446008ab407bb82882031 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 04:58:17.307626 22030 tablet_replica.cc:330] T 3192cb5fc49d40f8a940e03da41b405d P ebc08b93691f474698e93a5cc45baa6c: stopping tablet replica
I20260806 04:58:17.307634 22034 raft_consensus.cc:2258] T 3192cb5fc49d40f8a940e03da41b405d P 0c634c0504d446008ab407bb82882031 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 04:58:17.307833 22030 raft_consensus.cc:2229] T 3192cb5fc49d40f8a940e03da41b405d P ebc08b93691f474698e93a5cc45baa6c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 04:58:17.308058 22030 raft_consensus.cc:2258] T 3192cb5fc49d40f8a940e03da41b405d P ebc08b93691f474698e93a5cc45baa6c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 04:58:17.308328 22033 tablet_replica.cc:330] T 3192cb5fc49d40f8a940e03da41b405d P 8f45f7b26a8f43799df89071dca99b7a: stopping tablet replica
I20260806 04:58:17.308530 22033 raft_consensus.cc:2229] T 3192cb5fc49d40f8a940e03da41b405d P 8f45f7b26a8f43799df89071dca99b7a [term 1 LEADER]: Raft consensus shutting down.
I20260806 04:58:17.308816 9601 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu pid 21270
I20260806 04:58:17.308903 22033 raft_consensus.cc:2258] T 3192cb5fc49d40f8a940e03da41b405d P 8f45f7b26a8f43799df89071dca99b7a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 04:58:17.309168 22034 ts_tablet_manager.cc:1905] T 3192cb5fc49d40f8a940e03da41b405d P 0c634c0504d446008ab407bb82882031: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 04:58:17.309892 22030 ts_tablet_manager.cc:1905] T 3192cb5fc49d40f8a940e03da41b405d P ebc08b93691f474698e93a5cc45baa6c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 04:58:17.316299 22033 ts_tablet_manager.cc:1905] T 3192cb5fc49d40f8a940e03da41b405d P 8f45f7b26a8f43799df89071dca99b7a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 04:58:17.317993 22034 ts_tablet_manager.cc:1918] T 3192cb5fc49d40f8a940e03da41b405d P 0c634c0504d446008ab407bb82882031: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 04:58:17.318166 22034 log.cc:1196] T 3192cb5fc49d40f8a940e03da41b405d P 0c634c0504d446008ab407bb82882031: Deleting WAL directory at /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785992227050689-9601-0/minicluster-data/ts-1/wal/wals/3192cb5fc49d40f8a940e03da41b405d
I20260806 04:58:17.319285 22034 ts_tablet_manager.cc:1939] T 3192cb5fc49d40f8a940e03da41b405d P 0c634c0504d446008ab407bb82882031: Deleting consensus metadata
I20260806 04:58:17.320606 21150 catalog_manager.cc:4841] TS 0c634c0504d446008ab407bb82882031 (127.9.96.66:46591): tablet 3192cb5fc49d40f8a940e03da41b405d (table test-table [id=8962763aa3af43e396ca57444cb0ef0b]) successfully deleted
I20260806 04:58:17.324832 22030 ts_tablet_manager.cc:1918] T 3192cb5fc49d40f8a940e03da41b405d P ebc08b93691f474698e93a5cc45baa6c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 04:58:17.324993 22030 log.cc:1196] T 3192cb5fc49d40f8a940e03da41b405d P ebc08b93691f474698e93a5cc45baa6c: Deleting WAL directory at /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785992227050689-9601-0/minicluster-data/ts-2/wal/wals/3192cb5fc49d40f8a940e03da41b405d
I20260806 04:58:17.325464 22030 ts_tablet_manager.cc:1939] T 3192cb5fc49d40f8a940e03da41b405d P ebc08b93691f474698e93a5cc45baa6c: Deleting consensus metadata
I20260806 04:58:17.326570 21146 catalog_manager.cc:4841] TS ebc08b93691f474698e93a5cc45baa6c (127.9.96.67:36799): tablet 3192cb5fc49d40f8a940e03da41b405d (table test-table [id=8962763aa3af43e396ca57444cb0ef0b]) successfully deleted
I20260806 04:58:17.326741 22033 ts_tablet_manager.cc:1918] T 3192cb5fc49d40f8a940e03da41b405d P 8f45f7b26a8f43799df89071dca99b7a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 04:58:17.326880 22033 log.cc:1196] T 3192cb5fc49d40f8a940e03da41b405d P 8f45f7b26a8f43799df89071dca99b7a: Deleting WAL directory at /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785992227050689-9601-0/minicluster-data/ts-0/wal/wals/3192cb5fc49d40f8a940e03da41b405d
I20260806 04:58:17.327715 22033 ts_tablet_manager.cc:1939] T 3192cb5fc49d40f8a940e03da41b405d P 8f45f7b26a8f43799df89071dca99b7a: Deleting consensus metadata
Aug 06 04:58:17 dist-test-slave-x4qh krb5kdc[21106](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992296, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.9.96.65@KRBTEST.COM
I20260806 04:58:17.329010 21148 catalog_manager.cc:4841] TS 8f45f7b26a8f43799df89071dca99b7a (127.9.96.65:39027): tablet 3192cb5fc49d40f8a940e03da41b405d (table test-table [id=8962763aa3af43e396ca57444cb0ef0b]) successfully deleted
I20260806 04:58:17.336817 22010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:58:17.311663 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35832 (local address 127.9.96.65:39027)
0806 04:58:17.311854 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0806 04:58:17.311860 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 04:58:17.320285 (+ 8425us) server_negotiation.cc:407] Connection header received
0806 04:58:17.320760 (+ 475us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:58:17.320765 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:58:17.320857 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:58:17.320948 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:58:17.322408 (+ 1460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:17.323212 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:17.324530 (+ 1318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:17.325030 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:17.325169 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:58:17.329974 (+ 4805us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:58:17.329990 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:58:17.330007 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:58:17.330037 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:58:17.333582 (+ 3545us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:58:17.334320 (+ 738us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:58:17.334327 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:58:17.334329 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:58:17.334518 (+ 189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:58:17.334957 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:58:17.334967 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:58:17.334969 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:58:17.335480 (+ 511us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:58:17.335832 (+ 352us) server_negotiation.cc:1035] Waiting for connection context
0806 04:58:17.336162 (+ 330us) server_negotiation.cc:299] Negotiation successful
0806 04:58:17.336401 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260806 04:58:17.337543 21463 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 04:58:17.515363 9601 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu with pid 21270
I20260806 04:58:17.552654 9601 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu pid 21482
Aug 06 04:58:17 dist-test-slave-x4qh krb5kdc[21106](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992296, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.9.96.66@KRBTEST.COM
I20260806 04:58:17.568250 22074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:58:17.553500 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34054 (local address 127.9.96.66:46591)
0806 04:58:17.554273 (+ 773us) server_negotiation.cc:206] Beginning negotiation
0806 04:58:17.554278 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 04:58:17.554293 (+ 15us) server_negotiation.cc:407] Connection header received
0806 04:58:17.554347 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:58:17.554351 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:58:17.554410 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:58:17.554499 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:58:17.555928 (+ 1429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:17.556646 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:17.557850 (+ 1204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:17.558262 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:17.558358 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:58:17.562434 (+ 4076us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:58:17.562447 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:58:17.562462 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:58:17.562499 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:58:17.565430 (+ 2931us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:58:17.566089 (+ 659us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:58:17.566103 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:58:17.566106 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:58:17.566226 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:58:17.566621 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:58:17.566627 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:58:17.566631 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:58:17.567011 (+ 380us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:58:17.567322 (+ 311us) server_negotiation.cc:1035] Waiting for connection context
0806 04:58:17.567665 (+ 343us) server_negotiation.cc:299] Negotiation successful
0806 04:58:17.567899 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":683,"thread_start_us":540,"threads_started":1}
I20260806 04:58:17.568920 21689 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 04:58:17.691444 9601 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu with pid 21482
I20260806 04:58:17.722409 9601 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu pid 21701
Aug 06 04:58:17 dist-test-slave-x4qh krb5kdc[21106](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992296, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.9.96.67@KRBTEST.COM
I20260806 04:58:17.737699 22077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:58:17.723156 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33168 (local address 127.9.96.67:36799)
0806 04:58:17.723773 (+ 617us) server_negotiation.cc:206] Beginning negotiation
0806 04:58:17.723777 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 04:58:17.723793 (+ 16us) server_negotiation.cc:407] Connection header received
0806 04:58:17.723857 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:58:17.723861 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:58:17.723919 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:58:17.724016 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:58:17.725318 (+ 1302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:17.726022 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:17.727311 (+ 1289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:17.727870 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:17.727960 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:58:17.732036 (+ 4076us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:58:17.732051 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:58:17.732067 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:58:17.732097 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:58:17.734942 (+ 2845us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:58:17.735981 (+ 1039us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:58:17.735988 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:58:17.735991 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:58:17.736125 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:58:17.736456 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:58:17.736460 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:58:17.736462 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:58:17.736813 (+ 351us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:58:17.737056 (+ 243us) server_negotiation.cc:1035] Waiting for connection context
0806 04:58:17.737279 (+ 223us) server_negotiation.cc:299] Negotiation successful
0806 04:58:17.737442 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":512,"thread_start_us":404,"threads_started":1}
I20260806 04:58:17.738385 21897 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 04:58:17.834947 9601 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu with pid 21701
I20260806 04:58:17.858491 9601 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu pid 21124
I20260806 04:58:17.859793 21222 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 04:58:17.974406 9601 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu with pid 21124
2026-08-06T04:58:17Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (6300 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785992227050689-9601-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 04:58:18 dist-test-slave-x4qh krb5kdc[22083](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 04:58:18 dist-test-slave-x4qh krb5kdc[22083](info): set up 2 sockets
Aug 06 04:58:18 dist-test-slave-x4qh krb5kdc[22083](info): commencing operation
krb5kdc: starting...
W20260806 04:58:19.276113 9601 mini_kdc.cc:121] Time spent starting KDC: real 1.248s user 0.000s sys 0.028s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785992227050689-9601-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785992227050689-9601-0/krb5kdc/test-user.keytab.
Aug 06 04:58:19 dist-test-slave-x4qh krb5kdc[22083](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992299, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T04:58:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T04:58:19Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.96.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.96.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.96.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.126.keytab.
Entry for principal kudu/127.9.96.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.126.keytab.
I20260806 04:58:19.526841 9601 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
/tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785992227050689-9601-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785992227050689-9601-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785992227050689-9601-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785992227050689-9601-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.96.126:39341
--webserver_interface=127.9.96.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.126.keytab
--principal=kudu/127.9.96.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.96.84:43705
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.96.126:39341
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785992227050689-9601-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785992227050689-9601-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785992227050689-9601-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 04:58:19.856101 22100 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 04:58:19.856372 22100 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 04:58:19.856465 22100 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 04:58:19.863394 22100 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 04:58:19.863493 22100 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 04:58:19.863544 22100 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 04:58:19.863600 22100 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 04:58:19.863648 22100 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 04:58:19.873123 22100 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.96.84:43705
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785992227050689-9601-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785992227050689-9601-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.96.126:39341
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.126.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.96.126:39341
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785992227050689-9601-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.96.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785992227050689-9601-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 03:56:12 UTC on ccycloud.quasar-guijbm.root.comops.site
build id 4569618
ASAN enabled
I20260806 04:58:19.874330 22100 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 04:58:19.875667 22100 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 04:58:19.887765 22111 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:58:19.887856 22108 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:58:19.887882 22115 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:58:19.889108 22100 server_base.cc:1034] running on GCE node
I20260806 04:58:19.889833 22100 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 04:58:19.891667 22100 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 04:58:19.892904 22100 hybrid_clock.cc:648] HybridClock initialized: now 1785992299892858 us; error 59 us; skew 500 ppm
I20260806 04:58:19.893292 22100 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-08-06T04:58:19Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1982 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785992227050689-9601-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 04:58:20 dist-test-slave-x4qh krb5kdc[22124](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 04:58:20 dist-test-slave-x4qh krb5kdc[22124](info): set up 2 sockets
Aug 06 04:58:20 dist-test-slave-x4qh krb5kdc[22124](info): commencing operation
krb5kdc: starting...
W20260806 04:58:21.272598 9601 mini_kdc.cc:121] Time spent starting KDC: real 1.262s user 0.000s sys 0.030s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785992227050689-9601-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785992227050689-9601-0/krb5kdc/test-user.keytab.
Aug 06 04:58:21 dist-test-slave-x4qh krb5kdc[22124](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992301, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T04:58:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T04:58:21Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.96.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.96.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.96.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.126.keytab.
Entry for principal kudu/127.9.96.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.126.keytab.
I20260806 04:58:21.515044 9601 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
/tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785992227050689-9601-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785992227050689-9601-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785992227050689-9601-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785992227050689-9601-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.96.126:44517
--webserver_interface=127.9.96.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.126.keytab
--principal=kudu/127.9.96.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.96.84:42905
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.96.126:44517
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785992227050689-9601-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785992227050689-9601-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785992227050689-9601-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 04:58:21.843343 22142 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 04:58:21.843770 22142 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 04:58:21.843940 22142 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 04:58:21.851028 22142 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 04:58:21.851177 22142 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 04:58:21.851285 22142 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 04:58:21.851421 22142 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 04:58:21.851521 22142 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 04:58:21.860714 22142 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.96.84:42905
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785992227050689-9601-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785992227050689-9601-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.96.126:44517
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.126.keytab
--principal=kudu/127.9.96.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.96.126:44517
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785992227050689-9601-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.96.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785992227050689-9601-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 03:56:12 UTC on ccycloud.quasar-guijbm.root.comops.site
build id 4569618
ASAN enabled
I20260806 04:58:21.861994 22142 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 04:58:21.863299 22142 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 04:58:21.875422 22151 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:58:21.875451 22149 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:58:21.875699 22147 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:58:21.876317 22142 server_base.cc:1034] running on GCE node
I20260806 04:58:21.877081 22142 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 04:58:21.878460 22142 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 04:58:21.879724 22142 hybrid_clock.cc:648] HybridClock initialized: now 1785992301879685 us; error 47 us; skew 500 ppm
I20260806 04:58:21.880142 22142 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 04:58:21 dist-test-slave-x4qh krb5kdc[22124](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992301, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.96.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 04:58:21.887632 22142 init.cc:377] Logged in from keytab as kudu/127.9.96.126@KRBTEST.COM (short username kudu)
I20260806 04:58:21.889201 22142 webserver.cc:460] Webserver started at http://127.9.96.126:43241/ using document root <none> and password file <none>
I20260806 04:58:21.889926 22142 fs_manager.cc:359] Metadata directory not provided
I20260806 04:58:21.890086 22142 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 04:58:21.890421 22142 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 04:58:21.893484 22142 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785992227050689-9601-0/minicluster-data/master-0/data/instance:
uuid: "737cae188e2643f79ee3b8687dec928b"
format_stamp: "Formatted at 2026-08-06 04:58:21 on dist-test-slave-x4qh"
server_key: "c4612f2efcc556ee8ef6f481e101a6fc"
server_key_iv: "72fcc7f08d20a05504caaa6a240a872b"
server_key_version: "encryptionkey@0"
I20260806 04:58:21.894320 22142 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785992227050689-9601-0/minicluster-data/master-0/wal/instance:
uuid: "737cae188e2643f79ee3b8687dec928b"
format_stamp: "Formatted at 2026-08-06 04:58:21 on dist-test-slave-x4qh"
server_key: "c4612f2efcc556ee8ef6f481e101a6fc"
server_key_iv: "72fcc7f08d20a05504caaa6a240a872b"
server_key_version: "encryptionkey@0"
I20260806 04:58:21.899279 22142 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260806 04:58:21.902664 22160 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 04:58:21.904027 22142 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 04:58:21.904227 22142 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785992227050689-9601-0/minicluster-data/master-0/data,/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785992227050689-9601-0/minicluster-data/master-0/wal
uuid: "737cae188e2643f79ee3b8687dec928b"
format_stamp: "Formatted at 2026-08-06 04:58:21 on dist-test-slave-x4qh"
server_key: "c4612f2efcc556ee8ef6f481e101a6fc"
server_key_iv: "72fcc7f08d20a05504caaa6a240a872b"
server_key_version: "encryptionkey@0"
I20260806 04:58:21.904464 22142 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785992227050689-9601-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785992227050689-9601-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785992227050689-9601-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 04:58:21.919178 22142 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 04:58:21.922968 22142 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 04:58:21.923278 22142 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 04:58:21.942330 22142 rpc_server.cc:307] RPC server started. Bound to: 127.9.96.126:44517
I20260806 04:58:21.942371 22230 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.96.126:44517 every 8 connection(s)
I20260806 04:58:21.943667 22142 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785992227050689-9601-0/minicluster-data/master-0/data/info.pb
I20260806 04:58:21.947190 22231 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 04:58:21.948045 9601 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu as pid 22142
I20260806 04:58:21.948326 9601 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785992227050689-9601-0/minicluster-data/master-0/wal/instance
I20260806 04:58:21.949190 9601 external_mini_cluster.cc:1428] Setting key ee4b0504d6ef7cc4a4dcdeabcb2b8cd6
I20260806 04:58:21.961959 22231 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 737cae188e2643f79ee3b8687dec928b: Bootstrap starting.
Aug 06 04:58:21 dist-test-slave-x4qh krb5kdc[22124](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992301, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.96.126@KRBTEST.COM
I20260806 04:58:21.965533 22231 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 737cae188e2643f79ee3b8687dec928b: Neither blocks nor log segments found. Creating new log.
I20260806 04:58:21.966766 22231 log.cc:824] T 00000000000000000000000000000000 P 737cae188e2643f79ee3b8687dec928b: Log is configured to *not* fsync() on all Append() calls
I20260806 04:58:21.969584 22231 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 737cae188e2643f79ee3b8687dec928b: No bootstrap required, opened a new log
I20260806 04:58:21.972506 22236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:58:21.951247 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57714 (local address 127.9.96.126:44517)
0806 04:58:21.952129 (+ 882us) server_negotiation.cc:206] Beginning negotiation
0806 04:58:21.952144 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 04:58:21.952209 (+ 65us) server_negotiation.cc:407] Connection header received
0806 04:58:21.954287 (+ 2078us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:58:21.954313 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:58:21.954712 (+ 399us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:58:21.955074 (+ 362us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:58:21.956785 (+ 1711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:21.958067 (+ 1282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:21.959906 (+ 1839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:21.960547 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:21.960668 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:58:21.965254 (+ 4586us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:58:21.965261 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:58:21.965274 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:58:21.965307 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:58:21.968522 (+ 3215us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:58:21.969237 (+ 715us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:58:21.969243 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:58:21.969249 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:58:21.969410 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:58:21.969903 (+ 493us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:58:21.969907 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:58:21.969909 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:58:21.970441 (+ 532us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:58:21.970770 (+ 329us) server_negotiation.cc:1035] Waiting for connection context
0806 04:58:21.971174 (+ 404us) server_negotiation.cc:299] Negotiation successful
0806 04:58:21.971600 (+ 426us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":657,"thread_start_us":449,"threads_started":1}
I20260806 04:58:21.977252 22231 raft_consensus.cc:348] T 00000000000000000000000000000000 P 737cae188e2643f79ee3b8687dec928b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "737cae188e2643f79ee3b8687dec928b" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 44517 } }
I20260806 04:58:21.977641 22231 raft_consensus.cc:374] T 00000000000000000000000000000000 P 737cae188e2643f79ee3b8687dec928b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 04:58:21.977763 22231 raft_consensus.cc:729] T 00000000000000000000000000000000 P 737cae188e2643f79ee3b8687dec928b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 737cae188e2643f79ee3b8687dec928b, State: Initialized, Role: FOLLOWER
I20260806 04:58:21.978287 22231 consensus_queue.cc:260] T 00000000000000000000000000000000 P 737cae188e2643f79ee3b8687dec928b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "737cae188e2643f79ee3b8687dec928b" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 44517 } }
I20260806 04:58:21.978582 22231 raft_consensus.cc:388] T 00000000000000000000000000000000 P 737cae188e2643f79ee3b8687dec928b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 04:58:21.978726 22231 raft_consensus.cc:482] T 00000000000000000000000000000000 P 737cae188e2643f79ee3b8687dec928b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 04:58:21.978909 22231 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 737cae188e2643f79ee3b8687dec928b [term 0 FOLLOWER]: Advancing to term 1
I20260806 04:58:21.981509 22231 raft_consensus.cc:504] T 00000000000000000000000000000000 P 737cae188e2643f79ee3b8687dec928b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "737cae188e2643f79ee3b8687dec928b" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 44517 } }
I20260806 04:58:21.981988 22231 leader_election.cc:302] T 00000000000000000000000000000000 P 737cae188e2643f79ee3b8687dec928b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 737cae188e2643f79ee3b8687dec928b; no voters:
I20260806 04:58:21.982601 22231 leader_election.cc:288] T 00000000000000000000000000000000 P 737cae188e2643f79ee3b8687dec928b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 04:58:21.982766 22240 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 737cae188e2643f79ee3b8687dec928b [term 1 FOLLOWER]: Leader election won for term 1
I20260806 04:58:21.983500 22240 raft_consensus.cc:686] T 00000000000000000000000000000000 P 737cae188e2643f79ee3b8687dec928b [term 1 LEADER]: Becoming Leader. State: Replica: 737cae188e2643f79ee3b8687dec928b, State: Running, Role: LEADER
I20260806 04:58:21.984027 22240 consensus_queue.cc:237] T 00000000000000000000000000000000 P 737cae188e2643f79ee3b8687dec928b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "737cae188e2643f79ee3b8687dec928b" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 44517 } }
I20260806 04:58:21.984730 22231 sys_catalog.cc:564] T 00000000000000000000000000000000 P 737cae188e2643f79ee3b8687dec928b [sys.catalog]: configured and running, proceeding with master startup.
I20260806 04:58:21.987108 22242 sys_catalog.cc:455] T 00000000000000000000000000000000 P 737cae188e2643f79ee3b8687dec928b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 737cae188e2643f79ee3b8687dec928b. Latest consensus state: current_term: 1 leader_uuid: "737cae188e2643f79ee3b8687dec928b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "737cae188e2643f79ee3b8687dec928b" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 44517 } } }
I20260806 04:58:21.987190 22241 sys_catalog.cc:455] T 00000000000000000000000000000000 P 737cae188e2643f79ee3b8687dec928b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "737cae188e2643f79ee3b8687dec928b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "737cae188e2643f79ee3b8687dec928b" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 44517 } } }
I20260806 04:58:21.987592 22242 sys_catalog.cc:458] T 00000000000000000000000000000000 P 737cae188e2643f79ee3b8687dec928b [sys.catalog]: This master's current role is: LEADER
I20260806 04:58:21.987674 22241 sys_catalog.cc:458] T 00000000000000000000000000000000 P 737cae188e2643f79ee3b8687dec928b [sys.catalog]: This master's current role is: LEADER
W20260806 04:58:21.996484 22261 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 737cae188e2643f79ee3b8687dec928b: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 04:58:21.996661 22261 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 04:58:21.996837 22248 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 04:58:21.998420 22248 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 04:58:22.006666 22248 catalog_manager.cc:1347] Generated new cluster ID: ec6e2c2b029f410094b9606eb943274e
I20260806 04:58:22.006812 22248 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 04:58:22.020444 22248 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 04:58:22.021695 22248 catalog_manager.cc:1504] Loading token signing keys...
I20260806 04:58:22.026928 22248 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 737cae188e2643f79ee3b8687dec928b: Generated new TSK 0
I20260806 04:58:22.027887 22248 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.9.96.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.96.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.96.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.65.keytab.
Entry for principal kudu/127.9.96.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.65.keytab.
I20260806 04:58:22.087210 9601 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
/tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785992227050689-9601-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785992227050689-9601-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785992227050689-9601-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785992227050689-9601-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.96.65:0
--local_ip_for_outbound_sockets=127.9.96.65
--webserver_interface=127.9.96.65
--webserver_port=0
--tserver_master_addrs=127.9.96.126:44517
--keytab_file=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.65.keytab
--principal=kudu/127.9.96.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.96.84:42905
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785992227050689-9601-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785992227050689-9601-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785992227050689-9601-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 04:58:22.411167 22270 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 04:58:22.411469 22270 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 04:58:22.411707 22270 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 04:58:22.418579 22270 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 04:58:22.418725 22270 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 04:58:22.418905 22270 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.96.65
I20260806 04:58:22.428318 22270 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.96.84:42905
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785992227050689-9601-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785992227050689-9601-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.65.keytab
--principal=kudu/127.9.96.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.96.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785992227050689-9601-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.96.65
--webserver_port=0
--tserver_master_addrs=127.9.96.126:44517
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.96.65
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785992227050689-9601-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 03:56:12 UTC on ccycloud.quasar-guijbm.root.comops.site
build id 4569618
ASAN enabled
I20260806 04:58:22.429612 22270 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 04:58:22.430841 22270 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 04:58:22.444104 22277 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:58:22.444128 22278 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:58:22.444061 22280 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:58:22.444984 22270 server_base.cc:1034] running on GCE node
I20260806 04:58:22.445567 22270 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 04:58:22.446497 22270 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 04:58:22.447695 22270 hybrid_clock.cc:648] HybridClock initialized: now 1785992302447646 us; error 44 us; skew 500 ppm
I20260806 04:58:22.448020 22270 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 04:58:22 dist-test-slave-x4qh krb5kdc[22124](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992302, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.96.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 04:58:22.451196 22270 init.cc:377] Logged in from keytab as kudu/127.9.96.65@KRBTEST.COM (short username kudu)
I20260806 04:58:22.452688 22270 webserver.cc:460] Webserver started at http://127.9.96.65:34099/ using document root <none> and password file <none>
I20260806 04:58:22.453351 22270 fs_manager.cc:359] Metadata directory not provided
I20260806 04:58:22.453492 22270 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 04:58:22.453819 22270 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 04:58:22.456701 22270 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785992227050689-9601-0/minicluster-data/ts-0/data/instance:
uuid: "d99fcc5aa5d54543abab0bda29d16e2e"
format_stamp: "Formatted at 2026-08-06 04:58:22 on dist-test-slave-x4qh"
server_key: "a07e665aea28f0c36b9c9e2b8d408501"
server_key_iv: "6b8a052ba3c9fdf130eff04a127bf08d"
server_key_version: "encryptionkey@0"
I20260806 04:58:22.457553 22270 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785992227050689-9601-0/minicluster-data/ts-0/wal/instance:
uuid: "d99fcc5aa5d54543abab0bda29d16e2e"
format_stamp: "Formatted at 2026-08-06 04:58:22 on dist-test-slave-x4qh"
server_key: "a07e665aea28f0c36b9c9e2b8d408501"
server_key_iv: "6b8a052ba3c9fdf130eff04a127bf08d"
server_key_version: "encryptionkey@0"
I20260806 04:58:22.462579 22270 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260806 04:58:22.466053 22290 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 04:58:22.467268 22270 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 04:58:22.467484 22270 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785992227050689-9601-0/minicluster-data/ts-0/data,/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785992227050689-9601-0/minicluster-data/ts-0/wal
uuid: "d99fcc5aa5d54543abab0bda29d16e2e"
format_stamp: "Formatted at 2026-08-06 04:58:22 on dist-test-slave-x4qh"
server_key: "a07e665aea28f0c36b9c9e2b8d408501"
server_key_iv: "6b8a052ba3c9fdf130eff04a127bf08d"
server_key_version: "encryptionkey@0"
I20260806 04:58:22.467698 22270 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785992227050689-9601-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785992227050689-9601-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785992227050689-9601-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 04:58:22.484156 22270 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 04:58:22.488628 22270 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 04:58:22.488917 22270 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 04:58:22.489908 22270 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 04:58:22.491212 22270 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 04:58:22.491322 22270 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 04:58:22.491499 22270 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 04:58:22.491576 22270 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 04:58:22.525862 22270 rpc_server.cc:307] RPC server started. Bound to: 127.9.96.65:39867
I20260806 04:58:22.525913 22454 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.96.65:39867 every 8 connection(s)
I20260806 04:58:22.527405 22270 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785992227050689-9601-0/minicluster-data/ts-0/data/info.pb
I20260806 04:58:22.531009 9601 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu as pid 22270
I20260806 04:58:22.531260 9601 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785992227050689-9601-0/minicluster-data/ts-0/wal/instance
I20260806 04:58:22.532074 9601 external_mini_cluster.cc:1428] Setting key 8a544c70c002dae941b6b401a76aaf2b
Aug 06 04:58:22 dist-test-slave-x4qh krb5kdc[22124](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992302, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.96.65@KRBTEST.COM for kudu/127.9.96.126@KRBTEST.COM
I20260806 04:58:22.549988 22459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:58:22.530028 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.96.65:35325 (local address 127.9.96.126:44517)
0806 04:58:22.530487 (+ 459us) server_negotiation.cc:206] Beginning negotiation
0806 04:58:22.530491 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 04:58:22.531629 (+ 1138us) server_negotiation.cc:407] Connection header received
0806 04:58:22.533665 (+ 2036us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:58:22.533670 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:58:22.533760 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:58:22.533907 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:58:22.536047 (+ 2140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:22.536755 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:22.539319 (+ 2564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:22.539782 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:22.539899 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:58:22.544199 (+ 4300us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:58:22.544204 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:58:22.544207 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:58:22.544239 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:58:22.546798 (+ 2559us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:58:22.547620 (+ 822us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:58:22.547625 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:58:22.547635 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:58:22.547752 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:58:22.548202 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:58:22.548214 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:58:22.548216 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:58:22.548464 (+ 248us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:58:22.548630 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0806 04:58:22.549347 (+ 717us) server_negotiation.cc:299] Negotiation successful
0806 04:58:22.549578 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":345,"thread_start_us":251,"threads_started":1}
I20260806 04:58:22.550908 22460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:58:22.530504 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:44517 (local address 127.9.96.65:35325)
0806 04:58:22.531465 (+ 961us) negotiation.cc:107] Waiting for socket to connect
0806 04:58:22.531503 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0806 04:58:22.533394 (+ 1891us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:58:22.534204 (+ 810us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:58:22.534216 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:58:22.535004 (+ 788us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:58:22.535854 (+ 850us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:58:22.535884 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:58:22.536943 (+ 1059us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:58:22.536952 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:58:22.538990 (+ 2038us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:58:22.539008 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:58:22.540022 (+ 1014us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:58:22.540032 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:58:22.540364 (+ 332us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:58:22.541117 (+ 753us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:58:22.541157 (+ 40us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:58:22.543974 (+ 2817us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 04:58:22.546963 (+ 2989us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:58:22.546971 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:58:22.546987 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:58:22.547487 (+ 500us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:58:22.547876 (+ 389us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:58:22.547883 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:58:22.547886 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:58:22.548073 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:58:22.548603 (+ 530us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 04:58:22.548610 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 04:58:22.548941 (+ 331us) client_negotiation.cc:769] Sending connection context
0806 04:58:22.549225 (+ 284us) client_negotiation.cc:240] Negotiation successful
0806 04:58:22.549764 (+ 539us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":632,"thread_start_us":378,"threads_started":1}
I20260806 04:58:22.552618 22455 heartbeater.cc:347] Connected to a master server at 127.9.96.126:44517
I20260806 04:58:22.552929 22455 heartbeater.cc:464] Registering TS with master...
I20260806 04:58:22.553648 22455 heartbeater.cc:511] Master 127.9.96.126:44517 requested a full tablet report, sending...
I20260806 04:58:22.556105 22183 ts_manager.cc:194] Registered new tserver with Master: d99fcc5aa5d54543abab0bda29d16e2e (127.9.96.65:39867)
I20260806 04:58:22.558269 22183 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.96.65@KRBTEST.COM'} at 127.9.96.65:35325
WARNING: no policy specified for kudu/127.9.96.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.96.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.96.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.66.keytab.
Entry for principal kudu/127.9.96.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.66.keytab.
I20260806 04:58:22.590468 9601 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
/tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785992227050689-9601-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785992227050689-9601-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785992227050689-9601-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785992227050689-9601-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.96.66:0
--local_ip_for_outbound_sockets=127.9.96.66
--webserver_interface=127.9.96.66
--webserver_port=0
--tserver_master_addrs=127.9.96.126:44517
--keytab_file=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.66.keytab
--principal=kudu/127.9.96.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.96.84:42905
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785992227050689-9601-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785992227050689-9601-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785992227050689-9601-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 04:58:22.917765 22466 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 04:58:22.918017 22466 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 04:58:22.918107 22466 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 04:58:22.925354 22466 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 04:58:22.925455 22466 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 04:58:22.925647 22466 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.96.66
I20260806 04:58:22.935062 22466 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.96.84:42905
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785992227050689-9601-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785992227050689-9601-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.66.keytab
--principal=kudu/127.9.96.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.96.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785992227050689-9601-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.96.66
--webserver_port=0
--tserver_master_addrs=127.9.96.126:44517
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.96.66
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785992227050689-9601-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 03:56:12 UTC on ccycloud.quasar-guijbm.root.comops.site
build id 4569618
ASAN enabled
I20260806 04:58:22.936364 22466 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 04:58:22.937666 22466 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 04:58:22.955789 22475 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:58:22.955789 22473 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:58:22.956020 22472 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:58:22.960915 22466 server_base.cc:1034] running on GCE node
I20260806 04:58:22.961542 22466 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 04:58:22.962595 22466 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 04:58:22.963816 22466 hybrid_clock.cc:648] HybridClock initialized: now 1785992302963774 us; error 37 us; skew 500 ppm
I20260806 04:58:22.964138 22466 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 04:58:22 dist-test-slave-x4qh krb5kdc[22124](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992302, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.96.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 04:58:22.967353 22466 init.cc:377] Logged in from keytab as kudu/127.9.96.66@KRBTEST.COM (short username kudu)
I20260806 04:58:22.968986 22466 webserver.cc:460] Webserver started at http://127.9.96.66:44507/ using document root <none> and password file <none>
I20260806 04:58:22.969717 22466 fs_manager.cc:359] Metadata directory not provided
I20260806 04:58:22.969882 22466 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 04:58:22.970211 22466 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 04:58:22.973366 22466 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785992227050689-9601-0/minicluster-data/ts-1/data/instance:
uuid: "bca1fad7a5a642238a7606913519102e"
format_stamp: "Formatted at 2026-08-06 04:58:22 on dist-test-slave-x4qh"
server_key: "f16e4b88149927af19d3f060723f25fa"
server_key_iv: "5bfe3508e0ec7ad4415efaf21050f53a"
server_key_version: "encryptionkey@0"
I20260806 04:58:22.974138 22466 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785992227050689-9601-0/minicluster-data/ts-1/wal/instance:
uuid: "bca1fad7a5a642238a7606913519102e"
format_stamp: "Formatted at 2026-08-06 04:58:22 on dist-test-slave-x4qh"
server_key: "f16e4b88149927af19d3f060723f25fa"
server_key_iv: "5bfe3508e0ec7ad4415efaf21050f53a"
server_key_version: "encryptionkey@0"
I20260806 04:58:22.979189 22466 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260806 04:58:22.982604 22485 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 04:58:22.983989 22466 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 04:58:22.984153 22466 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785992227050689-9601-0/minicluster-data/ts-1/data,/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785992227050689-9601-0/minicluster-data/ts-1/wal
uuid: "bca1fad7a5a642238a7606913519102e"
format_stamp: "Formatted at 2026-08-06 04:58:22 on dist-test-slave-x4qh"
server_key: "f16e4b88149927af19d3f060723f25fa"
server_key_iv: "5bfe3508e0ec7ad4415efaf21050f53a"
server_key_version: "encryptionkey@0"
I20260806 04:58:22.984338 22466 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785992227050689-9601-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785992227050689-9601-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785992227050689-9601-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 04:58:23.005600 22466 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 04:58:23.010074 22466 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 04:58:23.010372 22466 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 04:58:23.011308 22466 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 04:58:23.012601 22466 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 04:58:23.012673 22466 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 04:58:23.012851 22466 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 04:58:23.012918 22466 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 04:58:23.046998 22466 rpc_server.cc:307] RPC server started. Bound to: 127.9.96.66:32795
I20260806 04:58:23.047091 22640 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.96.66:32795 every 8 connection(s)
I20260806 04:58:23.048405 22466 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785992227050689-9601-0/minicluster-data/ts-1/data/info.pb
I20260806 04:58:23.054066 9601 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu as pid 22466
I20260806 04:58:23.054266 9601 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785992227050689-9601-0/minicluster-data/ts-1/wal/instance
I20260806 04:58:23.054701 9601 external_mini_cluster.cc:1428] Setting key db4461a23eb30d8533f9da4a58150fd0
Aug 06 04:58:23 dist-test-slave-x4qh krb5kdc[22124](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992302, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.96.66@KRBTEST.COM for kudu/127.9.96.126@KRBTEST.COM
I20260806 04:58:23.070185 22645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:58:23.051014 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.96.66:40347 (local address 127.9.96.126:44517)
0806 04:58:23.051521 (+ 507us) server_negotiation.cc:206] Beginning negotiation
0806 04:58:23.051526 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 04:58:23.052501 (+ 975us) server_negotiation.cc:407] Connection header received
0806 04:58:23.054507 (+ 2006us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:58:23.054515 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:58:23.054605 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:58:23.054730 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:58:23.056657 (+ 1927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:23.057694 (+ 1037us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:23.059191 (+ 1497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:23.059983 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:23.060124 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:58:23.063466 (+ 3342us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:58:23.063475 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:58:23.063483 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:58:23.063528 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:58:23.067062 (+ 3534us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:58:23.067988 (+ 926us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:58:23.067991 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:58:23.067994 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:58:23.068109 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:58:23.068627 (+ 518us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:58:23.068630 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:58:23.068632 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:58:23.068905 (+ 273us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:58:23.069007 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0806 04:58:23.069776 (+ 769us) server_negotiation.cc:299] Negotiation successful
0806 04:58:23.069937 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":404,"thread_start_us":276,"threads_started":1}
I20260806 04:58:23.071703 22646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:58:23.051603 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:44517 (local address 127.9.96.66:40347)
0806 04:58:23.052334 (+ 731us) negotiation.cc:107] Waiting for socket to connect
0806 04:58:23.052380 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0806 04:58:23.054186 (+ 1806us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:58:23.054987 (+ 801us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:58:23.054995 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:58:23.055705 (+ 710us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:58:23.056468 (+ 763us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:58:23.056494 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:58:23.057858 (+ 1364us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:58:23.057867 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:58:23.059037 (+ 1170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:58:23.059048 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:58:23.060180 (+ 1132us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:58:23.060185 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:58:23.060369 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:58:23.061087 (+ 718us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:58:23.061109 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:58:23.063203 (+ 2094us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 04:58:23.067201 (+ 3998us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:58:23.067213 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:58:23.067228 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:58:23.067857 (+ 629us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:58:23.068227 (+ 370us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:58:23.068233 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:58:23.068237 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:58:23.068483 (+ 246us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:58:23.069037 (+ 554us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 04:58:23.069046 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 04:58:23.069422 (+ 376us) client_negotiation.cc:769] Sending connection context
0806 04:58:23.069687 (+ 265us) client_negotiation.cc:240] Negotiation successful
0806 04:58:23.070220 (+ 533us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":505,"thread_start_us":294,"threads_started":1}
I20260806 04:58:23.073225 22643 heartbeater.cc:347] Connected to a master server at 127.9.96.126:44517
I20260806 04:58:23.073520 22643 heartbeater.cc:464] Registering TS with master...
I20260806 04:58:23.074167 22643 heartbeater.cc:511] Master 127.9.96.126:44517 requested a full tablet report, sending...
I20260806 04:58:23.075690 22183 ts_manager.cc:194] Registered new tserver with Master: bca1fad7a5a642238a7606913519102e (127.9.96.66:32795)
I20260806 04:58:23.076995 22183 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.96.66@KRBTEST.COM'} at 127.9.96.66:40347
WARNING: no policy specified for kudu/127.9.96.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.96.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.96.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.67.keytab.
Entry for principal kudu/127.9.96.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.67.keytab.
I20260806 04:58:23.111588 9601 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
/tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785992227050689-9601-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785992227050689-9601-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785992227050689-9601-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785992227050689-9601-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.96.67:0
--local_ip_for_outbound_sockets=127.9.96.67
--webserver_interface=127.9.96.67
--webserver_port=0
--tserver_master_addrs=127.9.96.126:44517
--keytab_file=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.67.keytab
--principal=kudu/127.9.96.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.96.84:42905
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785992227050689-9601-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785992227050689-9601-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785992227050689-9601-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 04:58:23.439059 22651 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 04:58:23.439389 22651 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 04:58:23.439596 22651 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 04:58:23.446664 22651 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 04:58:23.446763 22651 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 04:58:23.446866 22651 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.96.67
I20260806 04:58:23.456211 22651 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.96.84:42905
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785992227050689-9601-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785992227050689-9601-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.67.keytab
--principal=kudu/127.9.96.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.96.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785992227050689-9601-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.96.67
--webserver_port=0
--tserver_master_addrs=127.9.96.126:44517
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.96.67
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785992227050689-9601-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 03:56:12 UTC on ccycloud.quasar-guijbm.root.comops.site
build id 4569618
ASAN enabled
I20260806 04:58:23.457320 22651 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 04:58:23.458530 22651 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 04:58:23.472309 22661 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:58:23.472255 22659 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:58:23.472250 22656 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:58:23.473462 22651 server_base.cc:1034] running on GCE node
I20260806 04:58:23.473961 22651 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 04:58:23.474764 22651 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 04:58:23.475960 22651 hybrid_clock.cc:648] HybridClock initialized: now 1785992303475905 us; error 48 us; skew 500 ppm
I20260806 04:58:23.476265 22651 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 04:58:23 dist-test-slave-x4qh krb5kdc[22124](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992303, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.96.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 04:58:23.479496 22651 init.cc:377] Logged in from keytab as kudu/127.9.96.67@KRBTEST.COM (short username kudu)
I20260806 04:58:23.480926 22651 webserver.cc:460] Webserver started at http://127.9.96.67:46559/ using document root <none> and password file <none>
I20260806 04:58:23.481575 22651 fs_manager.cc:359] Metadata directory not provided
I20260806 04:58:23.481743 22651 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 04:58:23.482146 22651 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 04:58:23.485091 22651 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785992227050689-9601-0/minicluster-data/ts-2/data/instance:
uuid: "bb8d9f5ceb91476f8b9a98a926ab776f"
format_stamp: "Formatted at 2026-08-06 04:58:23 on dist-test-slave-x4qh"
server_key: "f61863dac4f3c449c326de00958ed091"
server_key_iv: "e23d33aaf9174b82adb48bd8f48f91de"
server_key_version: "encryptionkey@0"
I20260806 04:58:23.486002 22651 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785992227050689-9601-0/minicluster-data/ts-2/wal/instance:
uuid: "bb8d9f5ceb91476f8b9a98a926ab776f"
format_stamp: "Formatted at 2026-08-06 04:58:23 on dist-test-slave-x4qh"
server_key: "f61863dac4f3c449c326de00958ed091"
server_key_iv: "e23d33aaf9174b82adb48bd8f48f91de"
server_key_version: "encryptionkey@0"
I20260806 04:58:23.491693 22651 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260806 04:58:23.494946 22671 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 04:58:23.496266 22651 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 04:58:23.496449 22651 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785992227050689-9601-0/minicluster-data/ts-2/data,/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785992227050689-9601-0/minicluster-data/ts-2/wal
uuid: "bb8d9f5ceb91476f8b9a98a926ab776f"
format_stamp: "Formatted at 2026-08-06 04:58:23 on dist-test-slave-x4qh"
server_key: "f61863dac4f3c449c326de00958ed091"
server_key_iv: "e23d33aaf9174b82adb48bd8f48f91de"
server_key_version: "encryptionkey@0"
I20260806 04:58:23.496662 22651 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785992227050689-9601-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785992227050689-9601-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785992227050689-9601-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 04:58:23.526856 22651 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 04:58:23.532044 22651 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 04:58:23.532384 22651 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 04:58:23.533259 22651 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 04:58:23.534472 22651 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 04:58:23.534545 22651 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 04:58:23.534624 22651 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 04:58:23.534668 22651 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 04:58:23.561559 22455 heartbeater.cc:503] Master 127.9.96.126:44517 was elected leader, sending a full tablet report...
I20260806 04:58:23.569379 22651 rpc_server.cc:307] RPC server started. Bound to: 127.9.96.67:34663
I20260806 04:58:23.569474 22827 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.96.67:34663 every 8 connection(s)
I20260806 04:58:23.570829 22651 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785992227050689-9601-0/minicluster-data/ts-2/data/info.pb
I20260806 04:58:23.573168 9601 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu as pid 22651
I20260806 04:58:23.573385 9601 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785992227050689-9601-0/minicluster-data/ts-2/wal/instance
I20260806 04:58:23.574050 9601 external_mini_cluster.cc:1428] Setting key dc3249f0eed9ee63e90cf42abfa4fabb
Aug 06 04:58:23 dist-test-slave-x4qh krb5kdc[22124](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992303, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.96.67@KRBTEST.COM for kudu/127.9.96.126@KRBTEST.COM
I20260806 04:58:23.594730 22830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:58:23.573575 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.96.67:52597 (local address 127.9.96.126:44517)
0806 04:58:23.579748 (+ 6173us) server_negotiation.cc:206] Beginning negotiation
0806 04:58:23.579752 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 04:58:23.579769 (+ 17us) server_negotiation.cc:407] Connection header received
0806 04:58:23.579859 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:58:23.579864 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:58:23.579966 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:58:23.580091 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:58:23.581767 (+ 1676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:23.582619 (+ 852us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:23.584478 (+ 1859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:23.585138 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:23.585236 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:58:23.588793 (+ 3557us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:58:23.588797 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:58:23.588800 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:58:23.588833 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:58:23.591260 (+ 2427us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:58:23.592128 (+ 868us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:58:23.592132 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:58:23.592134 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:58:23.592268 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:58:23.592775 (+ 507us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:58:23.592778 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:58:23.592780 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:58:23.593194 (+ 414us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:58:23.593342 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0806 04:58:23.594220 (+ 878us) server_negotiation.cc:299] Negotiation successful
0806 04:58:23.594454 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6066,"thread_start_us":5948,"threads_started":1}
I20260806 04:58:23.595700 22831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:58:23.574087 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:44517 (local address 127.9.96.67:52597)
0806 04:58:23.575373 (+ 1286us) negotiation.cc:107] Waiting for socket to connect
0806 04:58:23.575457 (+ 84us) client_negotiation.cc:174] Beginning negotiation
0806 04:58:23.578700 (+ 3243us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:58:23.580349 (+ 1649us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:58:23.580357 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:58:23.580936 (+ 579us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:58:23.581542 (+ 606us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:58:23.581559 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:58:23.582754 (+ 1195us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:58:23.582758 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:58:23.584283 (+ 1525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:58:23.584307 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:58:23.585337 (+ 1030us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:58:23.585342 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:58:23.585597 (+ 255us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:58:23.586309 (+ 712us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:58:23.586331 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:58:23.588579 (+ 2248us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 04:58:23.591464 (+ 2885us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:58:23.591471 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:58:23.591482 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:58:23.591898 (+ 416us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:58:23.592408 (+ 510us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:58:23.592413 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:58:23.592416 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:58:23.592627 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:58:23.593342 (+ 715us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 04:58:23.593350 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 04:58:23.593760 (+ 410us) client_negotiation.cc:769] Sending connection context
0806 04:58:23.594050 (+ 290us) client_negotiation.cc:240] Negotiation successful
0806 04:58:23.594570 (+ 520us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":901,"thread_start_us":635,"threads_started":1}
I20260806 04:58:23.596975 22828 heartbeater.cc:347] Connected to a master server at 127.9.96.126:44517
I20260806 04:58:23.597290 22828 heartbeater.cc:464] Registering TS with master...
I20260806 04:58:23.597996 22828 heartbeater.cc:511] Master 127.9.96.126:44517 requested a full tablet report, sending...
I20260806 04:58:23.599380 22183 ts_manager.cc:194] Registered new tserver with Master: bb8d9f5ceb91476f8b9a98a926ab776f (127.9.96.67:34663)
I20260806 04:58:23.600241 22183 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.96.67@KRBTEST.COM'} at 127.9.96.67:52597
I20260806 04:58:23.606361 9601 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.9.96.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.9.96.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.9.96.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785992227050689-9601-0/krb5kdc/oryx_127.9.96.65.keytab.
Entry for principal oryx/127.9.96.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785992227050689-9601-0/krb5kdc/oryx_127.9.96.65.keytab.
WARNING: no policy specified for oryx/127.9.96.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.9.96.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.9.96.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785992227050689-9601-0/krb5kdc/oryx_127.9.96.66.keytab.
Entry for principal oryx/127.9.96.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785992227050689-9601-0/krb5kdc/oryx_127.9.96.66.keytab.
WARNING: no policy specified for oryx/127.9.96.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.9.96.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.9.96.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785992227050689-9601-0/krb5kdc/oryx_127.9.96.67.keytab.
Entry for principal oryx/127.9.96.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785992227050689-9601-0/krb5kdc/oryx_127.9.96.67.keytab.
I20260806 04:58:23.750155 9601 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu pid 22270
Aug 06 04:58:23 dist-test-slave-x4qh krb5kdc[22124](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992301, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.96.65@KRBTEST.COM
I20260806 04:58:23.767081 22842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:58:23.751331 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33714 (local address 127.9.96.65:39867)
0806 04:58:23.752418 (+ 1087us) server_negotiation.cc:206] Beginning negotiation
0806 04:58:23.752424 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 04:58:23.752449 (+ 25us) server_negotiation.cc:407] Connection header received
0806 04:58:23.752513 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:58:23.752520 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:58:23.752726 (+ 206us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:58:23.752859 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:58:23.754322 (+ 1463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:23.755308 (+ 986us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:23.756556 (+ 1248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:23.757093 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:23.757193 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:58:23.761188 (+ 3995us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:58:23.761200 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:58:23.761214 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:58:23.761244 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:58:23.764216 (+ 2972us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:58:23.764964 (+ 748us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:58:23.764971 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:58:23.764973 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:58:23.765095 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:58:23.765497 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:58:23.765506 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:58:23.765508 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:58:23.765840 (+ 332us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:58:23.766156 (+ 316us) server_negotiation.cc:1035] Waiting for connection context
0806 04:58:23.766477 (+ 321us) server_negotiation.cc:299] Negotiation successful
0806 04:58:23.766734 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":943,"thread_start_us":749,"threads_started":1}
I20260806 04:58:23.768391 22449 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 04:58:23.858240 9601 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu with pid 22270
I20260806 04:58:23.884805 9601 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu pid 22466
Aug 06 04:58:23 dist-test-slave-x4qh krb5kdc[22124](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992301, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.96.66@KRBTEST.COM
I20260806 04:58:23.901432 22845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:58:23.885739 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50040 (local address 127.9.96.66:32795)
0806 04:58:23.886413 (+ 674us) server_negotiation.cc:206] Beginning negotiation
0806 04:58:23.886420 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 04:58:23.886447 (+ 27us) server_negotiation.cc:407] Connection header received
0806 04:58:23.886509 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:58:23.886515 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:58:23.886762 (+ 247us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:58:23.886921 (+ 159us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:58:23.888388 (+ 1467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:23.889597 (+ 1209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:23.890800 (+ 1203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:23.891331 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:23.891622 (+ 291us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:58:23.895776 (+ 4154us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:58:23.895787 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:58:23.895800 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:58:23.895840 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:58:23.898497 (+ 2657us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:58:23.899199 (+ 702us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:58:23.899212 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:58:23.899214 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:58:23.899345 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:58:23.899856 (+ 511us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:58:23.899861 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:58:23.899864 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:58:23.900193 (+ 329us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:58:23.900522 (+ 329us) server_negotiation.cc:1035] Waiting for connection context
0806 04:58:23.900819 (+ 297us) server_negotiation.cc:299] Negotiation successful
0806 04:58:23.901088 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":520,"thread_start_us":347,"threads_started":1}
I20260806 04:58:23.902676 22633 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 04:58:23.985520 9601 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu with pid 22466
I20260806 04:58:24.010967 9601 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu pid 22651
Aug 06 04:58:24 dist-test-slave-x4qh krb5kdc[22124](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992301, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.96.67@KRBTEST.COM
I20260806 04:58:24.027529 22847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:58:24.012003 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38870 (local address 127.9.96.67:34663)
0806 04:58:24.012541 (+ 538us) server_negotiation.cc:206] Beginning negotiation
0806 04:58:24.012549 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 04:58:24.012581 (+ 32us) server_negotiation.cc:407] Connection header received
0806 04:58:24.012647 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:58:24.012654 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:58:24.012921 (+ 267us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:58:24.013051 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:58:24.014369 (+ 1318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:24.015755 (+ 1386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:24.017155 (+ 1400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:24.017704 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:24.017807 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:58:24.022156 (+ 4349us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:58:24.022168 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:58:24.022182 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:58:24.022215 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:58:24.025157 (+ 2942us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:58:24.025810 (+ 653us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:58:24.025816 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:58:24.025818 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:58:24.025930 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:58:24.026255 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:58:24.026258 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:58:24.026260 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:58:24.026577 (+ 317us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:58:24.026818 (+ 241us) server_negotiation.cc:1035] Waiting for connection context
0806 04:58:24.026994 (+ 176us) server_negotiation.cc:299] Negotiation successful
0806 04:58:24.027173 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":390,"thread_start_us":253,"threads_started":1}
I20260806 04:58:24.028867 22820 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 04:58:24.111245 9601 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu with pid 22651
I20260806 04:58:24.133332 9601 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu pid 22142
I20260806 04:58:24.134496 22226 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 04:58:24.237010 9601 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu with pid 22142
I20260806 04:58:24.266517 9601 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
/tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785992227050689-9601-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785992227050689-9601-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785992227050689-9601-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785992227050689-9601-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.96.126:44517
--webserver_interface=127.9.96.126
--webserver_port=43241
--keytab_file=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.126.keytab
--principal=kudu/127.9.96.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.96.84:42905
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.96.126:44517
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785992227050689-9601-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785992227050689-9601-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785992227050689-9601-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 04:58:24.592543 22851 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 04:58:24.592873 22851 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 04:58:24.593036 22851 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 04:58:24.600323 22851 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 04:58:24.600476 22851 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 04:58:24.600601 22851 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 04:58:24.600716 22851 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 04:58:24.600796 22851 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 04:58:24.610666 22851 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.96.84:42905
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785992227050689-9601-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785992227050689-9601-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.96.126:44517
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.126.keytab
--principal=kudu/127.9.96.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.96.126:44517
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785992227050689-9601-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.96.126
--webserver_port=43241
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785992227050689-9601-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 03:56:12 UTC on ccycloud.quasar-guijbm.root.comops.site
build id 4569618
ASAN enabled
I20260806 04:58:24.612129 22851 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 04:58:24.613464 22851 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 04:58:24.625597 22858 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:58:24.625540 22860 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:58:24.625659 22857 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:58:24.626430 22851 server_base.cc:1034] running on GCE node
I20260806 04:58:24.627202 22851 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 04:58:24.628551 22851 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 04:58:24.629773 22851 hybrid_clock.cc:648] HybridClock initialized: now 1785992304629729 us; error 58 us; skew 500 ppm
I20260806 04:58:24.630120 22851 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 04:58:24 dist-test-slave-x4qh krb5kdc[22124](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992304, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.96.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 04:58:24.633304 22851 init.cc:377] Logged in from keytab as kudu/127.9.96.126@KRBTEST.COM (short username kudu)
I20260806 04:58:24.634725 22851 webserver.cc:460] Webserver started at http://127.9.96.126:43241/ using document root <none> and password file <none>
I20260806 04:58:24.635351 22851 fs_manager.cc:359] Metadata directory not provided
I20260806 04:58:24.635535 22851 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 04:58:24.640631 22851 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260806 04:58:24.643472 22870 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 04:58:24.644750 22851 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 04:58:24.644886 22851 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785992227050689-9601-0/minicluster-data/master-0/data,/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785992227050689-9601-0/minicluster-data/master-0/wal
uuid: "737cae188e2643f79ee3b8687dec928b"
format_stamp: "Formatted at 2026-08-06 04:58:21 on dist-test-slave-x4qh"
server_key: "c4612f2efcc556ee8ef6f481e101a6fc"
server_key_iv: "72fcc7f08d20a05504caaa6a240a872b"
server_key_version: "encryptionkey@0"
I20260806 04:58:24.645788 22851 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785992227050689-9601-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785992227050689-9601-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785992227050689-9601-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 04:58:24.675757 22851 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 04:58:24.679912 22851 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 04:58:24.680117 22851 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 04:58:24.698563 22851 rpc_server.cc:307] RPC server started. Bound to: 127.9.96.126:44517
I20260806 04:58:24.698603 22942 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.96.126:44517 every 8 connection(s)
I20260806 04:58:24.700012 22851 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785992227050689-9601-0/minicluster-data/master-0/data/info.pb
I20260806 04:58:24.706521 22945 sys_catalog.cc:263] Verifying existing consensus state
I20260806 04:58:24.709726 22945 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 737cae188e2643f79ee3b8687dec928b: Bootstrap starting.
I20260806 04:58:24.710424 9601 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu as pid 22851
I20260806 04:58:24.711218 9601 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
/tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785992227050689-9601-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785992227050689-9601-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785992227050689-9601-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785992227050689-9601-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.96.65:39867
--local_ip_for_outbound_sockets=127.9.96.65
--tserver_master_addrs=127.9.96.126:44517
--webserver_port=34099
--webserver_interface=127.9.96.65
--keytab_file=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.65.keytab
--principal=kudu/127.9.96.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.96.84:42905
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.9.96.65
--keytab_file=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785992227050689-9601-0/krb5kdc/oryx_127.9.96.65.keytab with env {KRB5CCNAME=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785992227050689-9601-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785992227050689-9601-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785992227050689-9601-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 04:58:24.728658 22945 log.cc:824] T 00000000000000000000000000000000 P 737cae188e2643f79ee3b8687dec928b: Log is configured to *not* fsync() on all Append() calls
I20260806 04:58:24.737160 22945 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 737cae188e2643f79ee3b8687dec928b: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 04:58:24.737679 22945 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 737cae188e2643f79ee3b8687dec928b: Bootstrap complete.
I20260806 04:58:24.746515 22945 raft_consensus.cc:348] T 00000000000000000000000000000000 P 737cae188e2643f79ee3b8687dec928b [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "737cae188e2643f79ee3b8687dec928b" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 44517 } }
I20260806 04:58:24.747331 22945 raft_consensus.cc:729] T 00000000000000000000000000000000 P 737cae188e2643f79ee3b8687dec928b [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 737cae188e2643f79ee3b8687dec928b, State: Initialized, Role: FOLLOWER
I20260806 04:58:24.747993 22945 consensus_queue.cc:260] T 00000000000000000000000000000000 P 737cae188e2643f79ee3b8687dec928b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "737cae188e2643f79ee3b8687dec928b" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 44517 } }
I20260806 04:58:24.748252 22945 raft_consensus.cc:388] T 00000000000000000000000000000000 P 737cae188e2643f79ee3b8687dec928b [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 04:58:24.748377 22945 raft_consensus.cc:482] T 00000000000000000000000000000000 P 737cae188e2643f79ee3b8687dec928b [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 04:58:24.748596 22945 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 737cae188e2643f79ee3b8687dec928b [term 1 FOLLOWER]: Advancing to term 2
I20260806 04:58:24.752593 22945 raft_consensus.cc:504] T 00000000000000000000000000000000 P 737cae188e2643f79ee3b8687dec928b [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "737cae188e2643f79ee3b8687dec928b" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 44517 } }
I20260806 04:58:24.753141 22945 leader_election.cc:302] T 00000000000000000000000000000000 P 737cae188e2643f79ee3b8687dec928b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 737cae188e2643f79ee3b8687dec928b; no voters:
I20260806 04:58:24.753820 22945 leader_election.cc:288] T 00000000000000000000000000000000 P 737cae188e2643f79ee3b8687dec928b [CANDIDATE]: Term 2 election: Requested vote from peers
I20260806 04:58:24.753952 22949 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 737cae188e2643f79ee3b8687dec928b [term 2 FOLLOWER]: Leader election won for term 2
I20260806 04:58:24.755090 22949 raft_consensus.cc:686] T 00000000000000000000000000000000 P 737cae188e2643f79ee3b8687dec928b [term 2 LEADER]: Becoming Leader. State: Replica: 737cae188e2643f79ee3b8687dec928b, State: Running, Role: LEADER
I20260806 04:58:24.755651 22949 consensus_queue.cc:237] T 00000000000000000000000000000000 P 737cae188e2643f79ee3b8687dec928b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "737cae188e2643f79ee3b8687dec928b" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 44517 } }
I20260806 04:58:24.756088 22945 sys_catalog.cc:564] T 00000000000000000000000000000000 P 737cae188e2643f79ee3b8687dec928b [sys.catalog]: configured and running, proceeding with master startup.
I20260806 04:58:24.758320 22951 sys_catalog.cc:455] T 00000000000000000000000000000000 P 737cae188e2643f79ee3b8687dec928b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 737cae188e2643f79ee3b8687dec928b. Latest consensus state: current_term: 2 leader_uuid: "737cae188e2643f79ee3b8687dec928b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "737cae188e2643f79ee3b8687dec928b" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 44517 } } }
I20260806 04:58:24.758762 22951 sys_catalog.cc:458] T 00000000000000000000000000000000 P 737cae188e2643f79ee3b8687dec928b [sys.catalog]: This master's current role is: LEADER
I20260806 04:58:24.759311 22950 sys_catalog.cc:455] T 00000000000000000000000000000000 P 737cae188e2643f79ee3b8687dec928b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "737cae188e2643f79ee3b8687dec928b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "737cae188e2643f79ee3b8687dec928b" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 44517 } } }
I20260806 04:58:24.759742 22950 sys_catalog.cc:458] T 00000000000000000000000000000000 P 737cae188e2643f79ee3b8687dec928b [sys.catalog]: This master's current role is: LEADER
I20260806 04:58:24.772475 22955 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 04:58:24.780035 22955 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 04:58:24.781839 22955 catalog_manager.cc:1259] Loaded cluster ID: ec6e2c2b029f410094b9606eb943274e
I20260806 04:58:24.781967 22955 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 04:58:24.787153 22955 catalog_manager.cc:1504] Loading token signing keys...
I20260806 04:58:24.789048 22955 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 737cae188e2643f79ee3b8687dec928b: Loaded TSK: 0
I20260806 04:58:24.790841 22955 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260806 04:58:25.083691 22947 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 04:58:25.084019 22947 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 04:58:25.084203 22947 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 04:58:25.091156 22947 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 04:58:25.091316 22947 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 04:58:25.091516 22947 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.96.65
I20260806 04:58:25.100939 22947 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.96.84:42905
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785992227050689-9601-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785992227050689-9601-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785992227050689-9601-0/krb5kdc/oryx_127.9.96.65.keytab
--principal=oryx/127.9.96.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.96.65:39867
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785992227050689-9601-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.96.65
--webserver_port=34099
--tserver_master_addrs=127.9.96.126:44517
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.96.65
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785992227050689-9601-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 03:56:12 UTC on ccycloud.quasar-guijbm.root.comops.site
build id 4569618
ASAN enabled
I20260806 04:58:25.102291 22947 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 04:58:25.104044 22947 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 04:58:25.125358 22975 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:58:25.126102 22977 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:58:25.126623 22980 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:58:25.127679 22947 server_base.cc:1034] running on GCE node
I20260806 04:58:25.128445 22947 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 04:58:25.129632 22947 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 04:58:25.130910 22947 hybrid_clock.cc:648] HybridClock initialized: now 1785992305130843 us; error 74 us; skew 500 ppm
I20260806 04:58:25.131320 22947 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 04:58:25 dist-test-slave-x4qh krb5kdc[22124](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992305, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.96.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 04:58:25.135200 22947 init.cc:377] Logged in from keytab as oryx/127.9.96.65@KRBTEST.COM (short username oryx)
I20260806 04:58:25.136801 22947 webserver.cc:460] Webserver started at http://127.9.96.65:34099/ using document root <none> and password file <none>
I20260806 04:58:25.137509 22947 fs_manager.cc:359] Metadata directory not provided
I20260806 04:58:25.137672 22947 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 04:58:25.143512 22947 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260806 04:58:25.146529 22990 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 04:58:25.148059 22947 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 04:58:25.148273 22947 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785992227050689-9601-0/minicluster-data/ts-0/data,/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785992227050689-9601-0/minicluster-data/ts-0/wal
uuid: "d99fcc5aa5d54543abab0bda29d16e2e"
format_stamp: "Formatted at 2026-08-06 04:58:22 on dist-test-slave-x4qh"
server_key: "a07e665aea28f0c36b9c9e2b8d408501"
server_key_iv: "6b8a052ba3c9fdf130eff04a127bf08d"
server_key_version: "encryptionkey@0"
I20260806 04:58:25.149405 22947 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785992227050689-9601-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785992227050689-9601-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785992227050689-9601-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 04:58:25.169152 22947 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 04:58:25.173453 22947 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 04:58:25.173736 22947 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 04:58:25.174757 22947 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 04:58:25.176172 22947 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 04:58:25.176343 22947 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 04:58:25.176492 22947 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 04:58:25.176602 22947 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 04:58:25.210951 22947 rpc_server.cc:307] RPC server started. Bound to: 127.9.96.65:39867
I20260806 04:58:25.211041 23145 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.96.65:39867 every 8 connection(s)
I20260806 04:58:25.212402 22947 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785992227050689-9601-0/minicluster-data/ts-0/data/info.pb
I20260806 04:58:25.217846 9601 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu as pid 22947
I20260806 04:58:25.218427 9601 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
/tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785992227050689-9601-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785992227050689-9601-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785992227050689-9601-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785992227050689-9601-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.96.66:32795
--local_ip_for_outbound_sockets=127.9.96.66
--tserver_master_addrs=127.9.96.126:44517
--webserver_port=44507
--webserver_interface=127.9.96.66
--keytab_file=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.66.keytab
--principal=kudu/127.9.96.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.96.84:42905
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.9.96.66
--keytab_file=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785992227050689-9601-0/krb5kdc/oryx_127.9.96.66.keytab with env {KRB5CCNAME=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785992227050689-9601-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785992227050689-9601-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785992227050689-9601-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 04:58:25 dist-test-slave-x4qh krb5kdc[22124](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.96.65@KRBTEST.COM for oryx/127.9.96.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 04:58:25 dist-test-slave-x4qh krb5kdc[22124](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.96.65@KRBTEST.COM for oryx/127.9.96.126@KRBTEST.COM, Server not found in Kerberos database
W20260806 04:58:25.231760 23150 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 04:58:25.215530 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.96.65:44541 (local address 127.9.96.126:44517)
0806 04:58:25.216354 (+ 824us) server_negotiation.cc:206] Beginning negotiation
0806 04:58:25.216369 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 04:58:25.216571 (+ 202us) server_negotiation.cc:407] Connection header received
0806 04:58:25.218967 (+ 2396us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:58:25.218978 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:58:25.219268 (+ 290us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:58:25.219683 (+ 415us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:58:25.221189 (+ 1506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:25.222440 (+ 1251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:25.224062 (+ 1622us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:25.224719 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:25.224928 (+ 209us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:58:25.230847 (+ 5919us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.96.65:44541: BlockingRecv error: recv got EOF from 127.9.96.65:44541 (error 108)
Metrics: {"server-negotiator.queue_time_us":547,"thread_start_us":317,"threads_started":1}
I20260806 04:58:25.231904 23152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:58:25.215546 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:44517 (local address 127.9.96.65:44541)
0806 04:58:25.216387 (+ 841us) negotiation.cc:107] Waiting for socket to connect
0806 04:58:25.216434 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0806 04:58:25.218333 (+ 1899us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:58:25.219867 (+ 1534us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:58:25.219882 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:58:25.220463 (+ 581us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:58:25.220980 (+ 517us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:58:25.220997 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:58:25.222632 (+ 1635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:58:25.222636 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:58:25.223887 (+ 1251us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:58:25.223899 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:58:25.224956 (+ 1057us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:58:25.224963 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:58:25.225240 (+ 277us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:58:25.225933 (+ 693us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:58:25.225953 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:58:25.230546 (+ 4593us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.96.126:44517: Server oryx/127.9.96.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":558,"thread_start_us":344,"threads_started":1}
W20260806 04:58:25.232637 23146 heartbeater.cc:644] Failed to heartbeat to 127.9.96.126:44517 (0 consecutive failures): Not authorized: Failed to ping master at 127.9.96.126:44517: Client connection negotiation failed: client connection to 127.9.96.126:44517: Server oryx/127.9.96.126@KRBTEST.COM not found in Kerberos database
Aug 06 04:58:25 dist-test-slave-x4qh krb5kdc[22124](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.96.65@KRBTEST.COM for oryx/127.9.96.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 04:58:25 dist-test-slave-x4qh krb5kdc[22124](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.96.65@KRBTEST.COM for oryx/127.9.96.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 04:58:25.241169 23152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:58:25.233292 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:44517 (local address 127.9.96.65:49013)
0806 04:58:25.233492 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0806 04:58:25.233511 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0806 04:58:25.233618 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:58:25.233939 (+ 321us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:58:25.233943 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:58:25.234274 (+ 331us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:58:25.234410 (+ 136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:58:25.234418 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:58:25.235477 (+ 1059us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:58:25.235482 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:58:25.236362 (+ 880us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:58:25.236373 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:58:25.237067 (+ 694us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:58:25.237073 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:58:25.237254 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:58:25.237915 (+ 661us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:58:25.237931 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:58:25.240896 (+ 2965us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.96.126:44517: Server oryx/127.9.96.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":66}
W20260806 04:58:25.241304 23150 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 04:58:25.233375 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.96.65:49013 (local address 127.9.96.126:44517)
0806 04:58:25.233531 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0806 04:58:25.233538 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 04:58:25.233555 (+ 17us) server_negotiation.cc:407] Connection header received
0806 04:58:25.233738 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:58:25.233743 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:58:25.233809 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:58:25.233914 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:58:25.234594 (+ 680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:25.235280 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:25.236509 (+ 1229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:25.236900 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:25.237027 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:58:25.241080 (+ 4053us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.96.65:49013: BlockingRecv error: recv got EOF from 127.9.96.65:49013 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
Aug 06 04:58:25 dist-test-slave-x4qh krb5kdc[22124](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.96.65@KRBTEST.COM for oryx/127.9.96.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 04:58:25 dist-test-slave-x4qh krb5kdc[22124](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.96.65@KRBTEST.COM for oryx/127.9.96.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 04:58:25.250757 23152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:58:25.242081 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:44517 (local address 127.9.96.65:51573)
0806 04:58:25.242277 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0806 04:58:25.242296 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0806 04:58:25.242392 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:58:25.242852 (+ 460us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:58:25.242855 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:58:25.243103 (+ 248us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:58:25.243231 (+ 128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:58:25.243238 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:58:25.244825 (+ 1587us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:58:25.244828 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:58:25.245698 (+ 870us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:58:25.245708 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:58:25.246832 (+ 1124us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:58:25.246836 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:58:25.246971 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:58:25.247619 (+ 648us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:58:25.247634 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:58:25.250558 (+ 2924us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.96.126:44517: Server oryx/127.9.96.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":78}
W20260806 04:58:25.251101 23146 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260806 04:58:25.251261 23150 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 04:58:25.242266 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.96.65:51573 (local address 127.9.96.126:44517)
0806 04:58:25.242483 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0806 04:58:25.242491 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 04:58:25.242516 (+ 25us) server_negotiation.cc:407] Connection header received
0806 04:58:25.242579 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:58:25.242585 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:58:25.242701 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:58:25.242835 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:58:25.243397 (+ 562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:25.244668 (+ 1271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:25.245859 (+ 1191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:25.246681 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:25.246843 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:58:25.250838 (+ 3995us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.96.65:51573: BlockingRecv error: recv got EOF from 127.9.96.65:51573 (error 108)
Metrics: {"server-negotiator.queue_time_us":73}
W20260806 04:58:25.560627 23154 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 04:58:25.560910 23154 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 04:58:25.561002 23154 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 04:58:25.569437 23154 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 04:58:25.569540 23154 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 04:58:25.569648 23154 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.96.66
I20260806 04:58:25.578898 23154 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.96.84:42905
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785992227050689-9601-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785992227050689-9601-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785992227050689-9601-0/krb5kdc/oryx_127.9.96.66.keytab
--principal=oryx/127.9.96.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.96.66:32795
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785992227050689-9601-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.96.66
--webserver_port=44507
--tserver_master_addrs=127.9.96.126:44517
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.96.66
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785992227050689-9601-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 03:56:12 UTC on ccycloud.quasar-guijbm.root.comops.site
build id 4569618
ASAN enabled
I20260806 04:58:25.580148 23154 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 04:58:25.581418 23154 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 04:58:25.594020 23162 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:58:25.594156 23165 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:58:25.594020 23161 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:58:25.595309 23154 server_base.cc:1034] running on GCE node
I20260806 04:58:25.595947 23154 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 04:58:25.596884 23154 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 04:58:25.598078 23154 hybrid_clock.cc:648] HybridClock initialized: now 1785992305598023 us; error 50 us; skew 500 ppm
I20260806 04:58:25.598387 23154 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 04:58:25 dist-test-slave-x4qh krb5kdc[22124](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992305, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.96.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 04:58:25.601537 23154 init.cc:377] Logged in from keytab as oryx/127.9.96.66@KRBTEST.COM (short username oryx)
I20260806 04:58:25.602880 23154 webserver.cc:460] Webserver started at http://127.9.96.66:44507/ using document root <none> and password file <none>
I20260806 04:58:25.603611 23154 fs_manager.cc:359] Metadata directory not provided
I20260806 04:58:25.603770 23154 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 04:58:25.608875 23154 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.002s
I20260806 04:58:25.611548 23173 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 04:58:25.612766 23154 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260806 04:58:25.612947 23154 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785992227050689-9601-0/minicluster-data/ts-1/data,/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785992227050689-9601-0/minicluster-data/ts-1/wal
uuid: "bca1fad7a5a642238a7606913519102e"
format_stamp: "Formatted at 2026-08-06 04:58:22 on dist-test-slave-x4qh"
server_key: "f16e4b88149927af19d3f060723f25fa"
server_key_iv: "5bfe3508e0ec7ad4415efaf21050f53a"
server_key_version: "encryptionkey@0"
I20260806 04:58:25.613876 23154 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785992227050689-9601-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785992227050689-9601-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785992227050689-9601-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 04:58:25.627892 23154 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 04:58:25.631548 23154 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 04:58:25.631831 23154 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 04:58:25.632711 23154 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 04:58:25.633962 23154 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 04:58:25.634087 23154 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 04:58:25.634227 23154 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 04:58:25.634316 23154 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 04:58:25.667272 23154 rpc_server.cc:307] RPC server started. Bound to: 127.9.96.66:32795
I20260806 04:58:25.667348 23336 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.96.66:32795 every 8 connection(s)
I20260806 04:58:25.668728 23154 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785992227050689-9601-0/minicluster-data/ts-1/data/info.pb
I20260806 04:58:25.676770 9601 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu as pid 23154
I20260806 04:58:25.677603 9601 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
/tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785992227050689-9601-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785992227050689-9601-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785992227050689-9601-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785992227050689-9601-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.96.67:34663
--local_ip_for_outbound_sockets=127.9.96.67
--tserver_master_addrs=127.9.96.126:44517
--webserver_port=46559
--webserver_interface=127.9.96.67
--keytab_file=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.67.keytab
--principal=kudu/127.9.96.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.96.84:42905
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.9.96.67
--keytab_file=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785992227050689-9601-0/krb5kdc/oryx_127.9.96.67.keytab with env {KRB5CCNAME=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785992227050689-9601-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785992227050689-9601-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785992227050689-9601-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 04:58:25 dist-test-slave-x4qh krb5kdc[22124](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.96.66@KRBTEST.COM for oryx/127.9.96.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 04:58:25 dist-test-slave-x4qh krb5kdc[22124](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.96.66@KRBTEST.COM for oryx/127.9.96.126@KRBTEST.COM, Server not found in Kerberos database
W20260806 04:58:25.686264 23150 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 04:58:25.671310 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.96.66:50997 (local address 127.9.96.126:44517)
0806 04:58:25.671503 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0806 04:58:25.671508 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 04:58:25.672834 (+ 1326us) server_negotiation.cc:407] Connection header received
0806 04:58:25.674533 (+ 1699us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:58:25.674537 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:58:25.674603 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:58:25.674689 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:58:25.676190 (+ 1501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:25.677100 (+ 910us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:25.679560 (+ 2460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:25.679948 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:25.680057 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:58:25.686088 (+ 6031us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.96.66:50997: BlockingRecv error: recv got EOF from 127.9.96.66:50997 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
I20260806 04:58:25.687191 23342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:58:25.671995 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:44517 (local address 127.9.96.66:50997)
0806 04:58:25.672676 (+ 681us) negotiation.cc:107] Waiting for socket to connect
0806 04:58:25.672712 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0806 04:58:25.674286 (+ 1574us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:58:25.674890 (+ 604us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:58:25.674896 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:58:25.675358 (+ 462us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:58:25.676036 (+ 678us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:58:25.676055 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:58:25.677270 (+ 1215us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:58:25.677277 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:58:25.679403 (+ 2126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:58:25.679428 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:58:25.680194 (+ 766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:58:25.680203 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:58:25.680536 (+ 333us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:58:25.681250 (+ 714us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:58:25.681282 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:58:25.685949 (+ 4667us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.96.126:44517: Server oryx/127.9.96.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":450,"thread_start_us":275,"threads_started":1}
W20260806 04:58:25.687819 23337 heartbeater.cc:644] Failed to heartbeat to 127.9.96.126:44517 (0 consecutive failures): Not authorized: Failed to ping master at 127.9.96.126:44517: Client connection negotiation failed: client connection to 127.9.96.126:44517: Server oryx/127.9.96.126@KRBTEST.COM not found in Kerberos database
Aug 06 04:58:25 dist-test-slave-x4qh krb5kdc[22124](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.96.66@KRBTEST.COM for oryx/127.9.96.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 04:58:25 dist-test-slave-x4qh krb5kdc[22124](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.96.66@KRBTEST.COM for oryx/127.9.96.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 04:58:25.695742 23342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:58:25.688447 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:44517 (local address 127.9.96.66:57573)
0806 04:58:25.688605 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0806 04:58:25.688621 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 04:58:25.688741 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:58:25.689059 (+ 318us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:58:25.689063 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:58:25.689378 (+ 315us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:58:25.689500 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:58:25.689508 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:58:25.690462 (+ 954us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:58:25.690467 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:58:25.691356 (+ 889us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:58:25.691419 (+ 63us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:58:25.692128 (+ 709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:58:25.692133 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:58:25.692307 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:58:25.692851 (+ 544us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:58:25.692868 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:58:25.695499 (+ 2631us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.96.126:44517: Server oryx/127.9.96.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":47}
W20260806 04:58:25.695928 23150 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 04:58:25.688618 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.96.66:57573 (local address 127.9.96.126:44517)
0806 04:58:25.688814 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0806 04:58:25.688820 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 04:58:25.688838 (+ 18us) server_negotiation.cc:407] Connection header received
0806 04:58:25.688885 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:58:25.688888 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:58:25.688937 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:58:25.689036 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:58:25.689656 (+ 620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:25.690327 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:25.691563 (+ 1236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:25.691972 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:25.692107 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:58:25.695688 (+ 3581us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.96.66:57573: BlockingRecv error: recv got EOF from 127.9.96.66:57573 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
Aug 06 04:58:25 dist-test-slave-x4qh krb5kdc[22124](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.96.66@KRBTEST.COM for oryx/127.9.96.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 04:58:25 dist-test-slave-x4qh krb5kdc[22124](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.96.66@KRBTEST.COM for oryx/127.9.96.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 04:58:25.706152 23342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:58:25.696512 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:44517 (local address 127.9.96.66:57413)
0806 04:58:25.696656 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0806 04:58:25.696673 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 04:58:25.696754 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:58:25.697084 (+ 330us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:58:25.697087 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:58:25.697329 (+ 242us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:58:25.697455 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:58:25.697462 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:58:25.698982 (+ 1520us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:58:25.698987 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:58:25.700463 (+ 1476us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:58:25.700478 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:58:25.701603 (+ 1125us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:58:25.701609 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:58:25.701808 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:58:25.702470 (+ 662us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:58:25.702490 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:58:25.705884 (+ 3394us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.96.126:44517: Server oryx/127.9.96.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":36}
W20260806 04:58:25.706604 23150 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 04:58:25.696609 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.96.66:57413 (local address 127.9.96.126:44517)
0806 04:58:25.696789 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0806 04:58:25.696796 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 04:58:25.696816 (+ 20us) server_negotiation.cc:407] Connection header received
0806 04:58:25.696865 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:58:25.696869 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:58:25.696947 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:58:25.697065 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:58:25.697599 (+ 534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:25.698810 (+ 1211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:25.700630 (+ 1820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:25.701433 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:25.701588 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:58:25.706177 (+ 4589us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.96.66:57413: BlockingRecv error: recv got EOF from 127.9.96.66:57413 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
W20260806 04:58:25.706646 23337 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260806 04:58:26.031270 23346 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 04:58:26.031661 23346 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 04:58:26.031840 23346 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 04:58:26.038957 23346 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 04:58:26.039106 23346 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 04:58:26.039273 23346 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.96.67
I20260806 04:58:26.048915 23346 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.96.84:42905
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785992227050689-9601-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785992227050689-9601-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785992227050689-9601-0/krb5kdc/oryx_127.9.96.67.keytab
--principal=oryx/127.9.96.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.96.67:34663
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785992227050689-9601-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.96.67
--webserver_port=46559
--tserver_master_addrs=127.9.96.126:44517
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.96.67
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785992227050689-9601-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 03:56:12 UTC on ccycloud.quasar-guijbm.root.comops.site
build id 4569618
ASAN enabled
I20260806 04:58:26.050282 23346 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 04:58:26.051604 23346 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 04:58:26.065102 23352 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:58:26.065121 23355 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:58:26.065126 23358 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:58:26.066224 23346 server_base.cc:1034] running on GCE node
I20260806 04:58:26.066809 23346 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 04:58:26.067778 23346 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 04:58:26.068998 23346 hybrid_clock.cc:648] HybridClock initialized: now 1785992306068957 us; error 35 us; skew 500 ppm
I20260806 04:58:26.069300 23346 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 04:58:26 dist-test-slave-x4qh krb5kdc[22124](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992306, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.96.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 04:58:26.072559 23346 init.cc:377] Logged in from keytab as oryx/127.9.96.67@KRBTEST.COM (short username oryx)
I20260806 04:58:26.073926 23346 webserver.cc:460] Webserver started at http://127.9.96.67:46559/ using document root <none> and password file <none>
I20260806 04:58:26.074568 23346 fs_manager.cc:359] Metadata directory not provided
I20260806 04:58:26.074723 23346 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 04:58:26.080027 23346 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.001s
I20260806 04:58:26.082584 23367 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 04:58:26.083889 23346 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260806 04:58:26.084067 23346 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785992227050689-9601-0/minicluster-data/ts-2/data,/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785992227050689-9601-0/minicluster-data/ts-2/wal
uuid: "bb8d9f5ceb91476f8b9a98a926ab776f"
format_stamp: "Formatted at 2026-08-06 04:58:23 on dist-test-slave-x4qh"
server_key: "f61863dac4f3c449c326de00958ed091"
server_key_iv: "e23d33aaf9174b82adb48bd8f48f91de"
server_key_version: "encryptionkey@0"
I20260806 04:58:26.085021 23346 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785992227050689-9601-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785992227050689-9601-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785992227050689-9601-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 04:58:26.110318 23346 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 04:58:26.114287 23346 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 04:58:26.114542 23346 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 04:58:26.115479 23346 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 04:58:26.116798 23346 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 04:58:26.116941 23346 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 04:58:26.117098 23346 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 04:58:26.117187 23346 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 04:58:26.150563 23346 rpc_server.cc:307] RPC server started. Bound to: 127.9.96.67:34663
I20260806 04:58:26.150663 23534 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.96.67:34663 every 8 connection(s)
I20260806 04:58:26.152036 23346 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785992227050689-9601-0/minicluster-data/ts-2/data/info.pb
I20260806 04:58:26.157789 9601 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu as pid 23346
Aug 06 04:58:26 dist-test-slave-x4qh krb5kdc[22124](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.96.67@KRBTEST.COM for oryx/127.9.96.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 04:58:26 dist-test-slave-x4qh krb5kdc[22124](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.96.67@KRBTEST.COM for oryx/127.9.96.126@KRBTEST.COM, Server not found in Kerberos database
W20260806 04:58:26.169649 23150 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 04:58:26.154716 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.96.67:45645 (local address 127.9.96.126:44517)
0806 04:58:26.154949 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0806 04:58:26.154955 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 04:58:26.156206 (+ 1251us) server_negotiation.cc:407] Connection header received
0806 04:58:26.158102 (+ 1896us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:58:26.158108 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:58:26.158199 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:58:26.158314 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:58:26.160781 (+ 2467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:26.161743 (+ 962us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:26.163339 (+ 1596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:26.163946 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:26.164034 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:58:26.169381 (+ 5347us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.96.67:45645: BlockingRecv error: recv got EOF from 127.9.96.67:45645 (error 108)
Metrics: {"mutex_wait_us":42,"server-negotiator.queue_time_us":86}
I20260806 04:58:26.170372 23537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:58:26.155207 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:44517 (local address 127.9.96.67:45645)
0806 04:58:26.156046 (+ 839us) negotiation.cc:107] Waiting for socket to connect
0806 04:58:26.156090 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0806 04:58:26.157802 (+ 1712us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:58:26.158562 (+ 760us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:58:26.158569 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:58:26.159442 (+ 873us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:58:26.160567 (+ 1125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:58:26.160598 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:58:26.161905 (+ 1307us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:58:26.161909 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:58:26.163198 (+ 1289us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:58:26.163212 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:58:26.164156 (+ 944us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:58:26.164165 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:58:26.164367 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:58:26.165078 (+ 711us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:58:26.165099 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:58:26.169199 (+ 4100us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.96.126:44517: Server oryx/127.9.96.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":521,"thread_start_us":336,"threads_started":1}
W20260806 04:58:26.171068 23535 heartbeater.cc:644] Failed to heartbeat to 127.9.96.126:44517 (0 consecutive failures): Not authorized: Failed to ping master at 127.9.96.126:44517: Client connection negotiation failed: client connection to 127.9.96.126:44517: Server oryx/127.9.96.126@KRBTEST.COM not found in Kerberos database
I20260806 04:58:26.175963 23539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:58:26.158786 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57726 (local address 127.9.96.126:44517)
0806 04:58:26.161133 (+ 2347us) server_negotiation.cc:206] Beginning negotiation
0806 04:58:26.161137 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 04:58:26.161150 (+ 13us) server_negotiation.cc:407] Connection header received
0806 04:58:26.161186 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:58:26.161189 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:58:26.161232 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:58:26.161350 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:58:26.163014 (+ 1664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:26.164048 (+ 1034us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:26.165347 (+ 1299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:26.165964 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:26.166091 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:58:26.168750 (+ 2659us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:58:26.168759 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:58:26.168776 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:58:26.168821 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:58:26.172787 (+ 3966us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:58:26.173375 (+ 588us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:58:26.173383 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:58:26.173390 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:58:26.173605 (+ 215us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:58:26.174043 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:58:26.174046 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:58:26.174047 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:58:26.174428 (+ 381us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:58:26.174625 (+ 197us) server_negotiation.cc:1035] Waiting for connection context
0806 04:58:26.175140 (+ 515us) server_negotiation.cc:299] Negotiation successful
0806 04:58:26.175534 (+ 394us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":2263,"thread_start_us":2139,"threads_started":1}
Aug 06 04:58:26 dist-test-slave-x4qh krb5kdc[22124](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.96.67@KRBTEST.COM for oryx/127.9.96.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 04:58:26 dist-test-slave-x4qh krb5kdc[22124](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.96.67@KRBTEST.COM for oryx/127.9.96.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 04:58:26.179970 23537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:58:26.171634 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:44517 (local address 127.9.96.67:54065)
0806 04:58:26.171912 (+ 278us) negotiation.cc:107] Waiting for socket to connect
0806 04:58:26.171926 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 04:58:26.172008 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:58:26.172292 (+ 284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:58:26.172295 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:58:26.172562 (+ 267us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:58:26.172682 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:58:26.172689 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:58:26.173761 (+ 1072us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:58:26.173767 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:58:26.174836 (+ 1069us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:58:26.174852 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:58:26.175756 (+ 904us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:58:26.175760 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:58:26.175886 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:58:26.176459 (+ 573us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:58:26.176474 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:58:26.179716 (+ 3242us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.96.126:44517: Server oryx/127.9.96.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":34}
W20260806 04:58:26.180024 23150 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 04:58:26.171729 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.96.67:54065 (local address 127.9.96.126:44517)
0806 04:58:26.171861 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0806 04:58:26.171866 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 04:58:26.172001 (+ 135us) server_negotiation.cc:407] Connection header received
0806 04:58:26.172108 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:58:26.172112 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:58:26.172179 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:58:26.172273 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:58:26.172833 (+ 560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:26.173612 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:26.174962 (+ 1350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:26.175615 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:26.175739 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:58:26.179875 (+ 4136us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.96.67:54065: BlockingRecv error: recv got EOF from 127.9.96.67:54065 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
Aug 06 04:58:26 dist-test-slave-x4qh krb5kdc[22124](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.96.67@KRBTEST.COM for oryx/127.9.96.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 04:58:26 dist-test-slave-x4qh krb5kdc[22124](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.96.67@KRBTEST.COM for oryx/127.9.96.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 04:58:26.187323 23537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:58:26.180640 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:44517 (local address 127.9.96.67:60671)
0806 04:58:26.180834 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0806 04:58:26.180853 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0806 04:58:26.180941 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:58:26.181188 (+ 247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:58:26.181190 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:58:26.181446 (+ 256us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:58:26.181572 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:58:26.181579 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:58:26.182511 (+ 932us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:58:26.182514 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:58:26.183359 (+ 845us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:58:26.183437 (+ 78us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:58:26.184018 (+ 581us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:58:26.184022 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:58:26.184133 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:58:26.184600 (+ 467us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:58:26.184616 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:58:26.187102 (+ 2486us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.96.126:44517: Server oryx/127.9.96.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":59}
W20260806 04:58:26.187656 23535 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260806 04:58:26.187700 23150 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 04:58:26.180729 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.96.67:60671 (local address 127.9.96.126:44517)
0806 04:58:26.180950 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0806 04:58:26.180954 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 04:58:26.180989 (+ 35us) server_negotiation.cc:407] Connection header received
0806 04:58:26.181023 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:58:26.181026 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:58:26.181083 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:58:26.181151 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:58:26.181689 (+ 538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:26.182390 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:26.183541 (+ 1151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:26.183907 (+ 366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:26.183992 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:58:26.187277 (+ 3285us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.96.67:60671: BlockingRecv error: recv got EOF from 127.9.96.67:60671 (error 108)
Metrics: {"server-negotiator.queue_time_us":142}
Aug 06 04:58:26 dist-test-slave-x4qh krb5kdc[22124](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.96.65@KRBTEST.COM for oryx/127.9.96.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 04:58:26 dist-test-slave-x4qh krb5kdc[22124](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.96.65@KRBTEST.COM for oryx/127.9.96.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 04:58:26.260190 23541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:58:26.251651 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:44517 (local address 127.9.96.65:34209)
0806 04:58:26.252206 (+ 555us) negotiation.cc:107] Waiting for socket to connect
0806 04:58:26.252222 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 04:58:26.252341 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:58:26.252632 (+ 291us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:58:26.252635 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:58:26.253161 (+ 526us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:58:26.253416 (+ 255us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:58:26.253427 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:58:26.254586 (+ 1159us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:58:26.254591 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:58:26.255647 (+ 1056us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:58:26.255659 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:58:26.256396 (+ 737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:58:26.256400 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:58:26.256523 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:58:26.257085 (+ 562us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:58:26.257102 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:58:26.259967 (+ 2865us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.96.126:44517: Server oryx/127.9.96.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":434,"thread_start_us":317,"threads_started":1}
W20260806 04:58:26.260514 23150 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 04:58:26.251755 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.96.65:34209 (local address 127.9.96.126:44517)
0806 04:58:26.251924 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0806 04:58:26.251929 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 04:58:26.252296 (+ 367us) server_negotiation.cc:407] Connection header received
0806 04:58:26.252439 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:58:26.252443 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:58:26.252502 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:58:26.252585 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:58:26.253550 (+ 965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:26.254413 (+ 863us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:26.255764 (+ 1351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:26.256247 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:26.256339 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:58:26.260166 (+ 3827us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.96.65:34209: BlockingRecv error: recv got EOF from 127.9.96.65:34209 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
Aug 06 04:58:26 dist-test-slave-x4qh krb5kdc[22124](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.96.66@KRBTEST.COM for oryx/127.9.96.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 04:58:26 dist-test-slave-x4qh krb5kdc[22124](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.96.66@KRBTEST.COM for oryx/127.9.96.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 04:58:26.715930 23542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:58:26.707283 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:44517 (local address 127.9.96.66:56225)
0806 04:58:26.707815 (+ 532us) negotiation.cc:107] Waiting for socket to connect
0806 04:58:26.707832 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 04:58:26.707941 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:58:26.708227 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:58:26.708230 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:58:26.708718 (+ 488us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:58:26.708911 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:58:26.708920 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:58:26.709902 (+ 982us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:58:26.709906 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:58:26.710897 (+ 991us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:58:26.710909 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:58:26.711837 (+ 928us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:58:26.711844 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:58:26.712049 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:58:26.712753 (+ 704us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:58:26.712771 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:58:26.715651 (+ 2880us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.96.126:44517: Server oryx/127.9.96.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":413,"thread_start_us":303,"threads_started":1}
W20260806 04:58:26.716063 23150 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 04:58:26.707490 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.96.66:56225 (local address 127.9.96.126:44517)
0806 04:58:26.707676 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0806 04:58:26.707680 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 04:58:26.707906 (+ 226us) server_negotiation.cc:407] Connection header received
0806 04:58:26.708036 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:58:26.708040 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:58:26.708098 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:58:26.708181 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:58:26.709037 (+ 856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:26.709775 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:26.711035 (+ 1260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:26.711623 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:26.711751 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:58:26.715842 (+ 4091us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.96.66:56225: BlockingRecv error: recv got EOF from 127.9.96.66:56225 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
Aug 06 04:58:27 dist-test-slave-x4qh krb5kdc[22124](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.96.67@KRBTEST.COM for oryx/127.9.96.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 04:58:27 dist-test-slave-x4qh krb5kdc[22124](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.96.67@KRBTEST.COM for oryx/127.9.96.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 04:58:27.201633 23543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:58:27.188276 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:44517 (local address 127.9.96.67:53083)
0806 04:58:27.188785 (+ 509us) negotiation.cc:107] Waiting for socket to connect
0806 04:58:27.188801 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 04:58:27.188910 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:58:27.189239 (+ 329us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:58:27.189242 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:58:27.189754 (+ 512us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:58:27.189949 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:58:27.189958 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:58:27.191006 (+ 1048us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:58:27.191010 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:58:27.192063 (+ 1053us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:58:27.192075 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:58:27.192736 (+ 661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:58:27.192740 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:58:27.192856 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:58:27.198019 (+ 5163us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:58:27.198036 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:58:27.201311 (+ 3275us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.96.126:44517: Server oryx/127.9.96.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":411,"thread_start_us":303,"threads_started":1}
W20260806 04:58:27.201794 23150 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 04:58:27.188294 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.96.67:53083 (local address 127.9.96.126:44517)
0806 04:58:27.188509 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0806 04:58:27.188519 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 04:58:27.188881 (+ 362us) server_negotiation.cc:407] Connection header received
0806 04:58:27.189042 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:58:27.189047 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:58:27.189121 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:58:27.189198 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:58:27.190078 (+ 880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:27.190836 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:27.192191 (+ 1355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:27.192587 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:27.192678 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:58:27.201535 (+ 8857us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.96.67:53083: BlockingRecv error: recv got EOF from 127.9.96.67:53083 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
Aug 06 04:58:27 dist-test-slave-x4qh krb5kdc[22124](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.96.65@KRBTEST.COM for oryx/127.9.96.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 04:58:27 dist-test-slave-x4qh krb5kdc[22124](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.96.65@KRBTEST.COM for oryx/127.9.96.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 04:58:27.268945 23544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:58:27.261087 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:44517 (local address 127.9.96.65:44197)
0806 04:58:27.261588 (+ 501us) negotiation.cc:107] Waiting for socket to connect
0806 04:58:27.261605 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 04:58:27.261722 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:58:27.262019 (+ 297us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:58:27.262023 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:58:27.262444 (+ 421us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:58:27.262590 (+ 146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:58:27.262599 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:58:27.263580 (+ 981us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:58:27.263584 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:58:27.264501 (+ 917us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:58:27.264512 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:58:27.265147 (+ 635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:58:27.265151 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:58:27.265262 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:58:27.265826 (+ 564us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:58:27.265842 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:58:27.268704 (+ 2862us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.96.126:44517: Server oryx/127.9.96.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":407,"thread_start_us":292,"threads_started":1}
W20260806 04:58:27.269117 23150 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 04:58:27.261178 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.96.65:44197 (local address 127.9.96.126:44517)
0806 04:58:27.261436 (+ 258us) server_negotiation.cc:206] Beginning negotiation
0806 04:58:27.261442 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 04:58:27.261686 (+ 244us) server_negotiation.cc:407] Connection header received
0806 04:58:27.261818 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:58:27.261821 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:58:27.261892 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:58:27.261971 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:58:27.262709 (+ 738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:27.263465 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:27.264622 (+ 1157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:27.265013 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:27.265108 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:58:27.268895 (+ 3787us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.96.65:44197: BlockingRecv error: recv got EOF from 127.9.96.65:44197 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
Aug 06 04:58:27 dist-test-slave-x4qh krb5kdc[22124](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.96.66@KRBTEST.COM for oryx/127.9.96.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 04:58:27 dist-test-slave-x4qh krb5kdc[22124](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.96.66@KRBTEST.COM for oryx/127.9.96.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 04:58:27.725505 23546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:58:27.716986 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:44517 (local address 127.9.96.66:60737)
0806 04:58:27.717556 (+ 570us) negotiation.cc:107] Waiting for socket to connect
0806 04:58:27.717572 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 04:58:27.717729 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:58:27.718110 (+ 381us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:58:27.718115 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:58:27.718585 (+ 470us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:58:27.718732 (+ 147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:58:27.718741 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:58:27.719841 (+ 1100us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:58:27.719847 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:58:27.720714 (+ 867us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:58:27.720726 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:58:27.721508 (+ 782us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:58:27.721516 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:58:27.721689 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:58:27.722407 (+ 718us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:58:27.722437 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:58:27.725163 (+ 2726us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.96.126:44517: Server oryx/127.9.96.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":480,"thread_start_us":349,"threads_started":1}
W20260806 04:58:27.725600 23150 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 04:58:27.717039 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.96.66:60737 (local address 127.9.96.126:44517)
0806 04:58:27.717348 (+ 309us) server_negotiation.cc:206] Beginning negotiation
0806 04:58:27.717356 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 04:58:27.717673 (+ 317us) server_negotiation.cc:407] Connection header received
0806 04:58:27.717881 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:58:27.717890 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:58:27.717951 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:58:27.718065 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:58:27.718889 (+ 824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:27.719696 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:27.720909 (+ 1213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:27.721334 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:27.721472 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:58:27.725377 (+ 3905us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.96.66:60737: BlockingRecv error: recv got EOF from 127.9.96.66:60737 (error 108)
Metrics: {"server-negotiator.queue_time_us":81}
Aug 06 04:58:28 dist-test-slave-x4qh krb5kdc[22124](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.96.67@KRBTEST.COM for oryx/127.9.96.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 04:58:28 dist-test-slave-x4qh krb5kdc[22124](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.96.67@KRBTEST.COM for oryx/127.9.96.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 04:58:28.211257 23548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:58:28.202647 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:44517 (local address 127.9.96.67:51665)
0806 04:58:28.203212 (+ 565us) negotiation.cc:107] Waiting for socket to connect
0806 04:58:28.203229 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 04:58:28.203410 (+ 181us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:58:28.203785 (+ 375us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:58:28.203790 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:58:28.204239 (+ 449us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:58:28.204406 (+ 167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:58:28.204415 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:58:28.205473 (+ 1058us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:58:28.205479 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:58:28.206379 (+ 900us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:58:28.206390 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:58:28.207095 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:58:28.207101 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:58:28.207267 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:58:28.207984 (+ 717us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:58:28.208002 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:58:28.210945 (+ 2943us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.96.126:44517: Server oryx/127.9.96.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":466,"thread_start_us":341,"threads_started":1}
W20260806 04:58:28.211468 23150 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 04:58:28.202880 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.96.67:51665 (local address 127.9.96.126:44517)
0806 04:58:28.203097 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0806 04:58:28.203105 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 04:58:28.203324 (+ 219us) server_negotiation.cc:407] Connection header received
0806 04:58:28.203542 (+ 218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:58:28.203547 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:58:28.203630 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:58:28.203740 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:58:28.204558 (+ 818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:28.205322 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:28.206523 (+ 1201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:28.206939 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:28.207077 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:58:28.211146 (+ 4069us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.96.67:51665: BlockingRecv error: recv got EOF from 127.9.96.67:51665 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
Aug 06 04:58:28 dist-test-slave-x4qh krb5kdc[22124](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.96.65@KRBTEST.COM for oryx/127.9.96.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 04:58:28 dist-test-slave-x4qh krb5kdc[22124](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.96.65@KRBTEST.COM for oryx/127.9.96.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 04:58:28.278285 23549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:58:28.270077 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:44517 (local address 127.9.96.65:55101)
0806 04:58:28.270653 (+ 576us) negotiation.cc:107] Waiting for socket to connect
0806 04:58:28.270670 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 04:58:28.270781 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:58:28.271084 (+ 303us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:58:28.271088 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:58:28.271457 (+ 369us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:58:28.271623 (+ 166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:58:28.271632 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:58:28.272632 (+ 1000us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:58:28.272635 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:58:28.273524 (+ 889us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:58:28.273535 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:58:28.274143 (+ 608us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:58:28.274146 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:58:28.274266 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:58:28.274850 (+ 584us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:58:28.274873 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:58:28.278041 (+ 3168us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.96.126:44517: Server oryx/127.9.96.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":449,"thread_start_us":324,"threads_started":1}
W20260806 04:58:28.278649 23150 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 04:58:28.270174 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.96.65:55101 (local address 127.9.96.126:44517)
0806 04:58:28.270346 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0806 04:58:28.270350 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 04:58:28.270743 (+ 393us) server_negotiation.cc:407] Connection header received
0806 04:58:28.270891 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:58:28.270894 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:58:28.270954 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:58:28.271037 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:58:28.271750 (+ 713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:28.272496 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:28.273642 (+ 1146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:28.274025 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:28.274110 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:58:28.278228 (+ 4118us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.96.65:55101: BlockingRecv error: recv got EOF from 127.9.96.65:55101 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
Aug 06 04:58:28 dist-test-slave-x4qh krb5kdc[22124](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.96.66@KRBTEST.COM for oryx/127.9.96.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 04:58:28 dist-test-slave-x4qh krb5kdc[22124](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.96.66@KRBTEST.COM for oryx/127.9.96.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 04:58:28.735042 23551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:58:28.726471 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:44517 (local address 127.9.96.66:34957)
0806 04:58:28.726960 (+ 489us) negotiation.cc:107] Waiting for socket to connect
0806 04:58:28.726975 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 04:58:28.727131 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:58:28.727554 (+ 423us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:58:28.727561 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:58:28.727929 (+ 368us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:58:28.728118 (+ 189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:58:28.728127 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:58:28.729243 (+ 1116us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:58:28.729248 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:58:28.730146 (+ 898us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:58:28.730158 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:58:28.730877 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:58:28.730883 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:58:28.731077 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:58:28.731781 (+ 704us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:58:28.731800 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:58:28.734686 (+ 2886us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.96.126:44517: Server oryx/127.9.96.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":389,"thread_start_us":252,"threads_started":1}
W20260806 04:58:28.735128 23150 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 04:58:28.726551 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.96.66:34957 (local address 127.9.96.126:44517)
0806 04:58:28.726780 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0806 04:58:28.726788 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 04:58:28.727059 (+ 271us) server_negotiation.cc:407] Connection header received
0806 04:58:28.727266 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:58:28.727276 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:58:28.727335 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:58:28.727501 (+ 166us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:58:28.728270 (+ 769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:28.729087 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:28.730302 (+ 1215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:28.730717 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:28.730865 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:58:28.734875 (+ 4010us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.96.66:34957: BlockingRecv error: recv got EOF from 127.9.96.66:34957 (error 108)
Metrics: {"server-negotiator.queue_time_us":76}
Aug 06 04:58:29 dist-test-slave-x4qh krb5kdc[22124](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.96.67@KRBTEST.COM for oryx/127.9.96.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 04:58:29 dist-test-slave-x4qh krb5kdc[22124](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.96.67@KRBTEST.COM for oryx/127.9.96.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 04:58:29.220603 23555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:58:29.212165 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:44517 (local address 127.9.96.67:34851)
0806 04:58:29.212752 (+ 587us) negotiation.cc:107] Waiting for socket to connect
0806 04:58:29.212768 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 04:58:29.212873 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:58:29.213234 (+ 361us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:58:29.213238 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:58:29.213584 (+ 346us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:58:29.213747 (+ 163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:58:29.213756 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:58:29.214695 (+ 939us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:58:29.214698 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:58:29.215567 (+ 869us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:58:29.215578 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:58:29.216341 (+ 763us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:58:29.216347 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:58:29.216489 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:58:29.217112 (+ 623us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:58:29.217129 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:58:29.220267 (+ 3138us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.96.126:44517: Server oryx/127.9.96.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":458,"thread_start_us":307,"threads_started":1}
W20260806 04:58:29.220691 23150 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 04:58:29.212305 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.96.67:34851 (local address 127.9.96.126:44517)
0806 04:58:29.212496 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0806 04:58:29.212502 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 04:58:29.212847 (+ 345us) server_negotiation.cc:407] Connection header received
0806 04:58:29.212973 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:58:29.212978 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:58:29.213052 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:58:29.213154 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:58:29.213913 (+ 759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:29.214589 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:29.215689 (+ 1100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:29.216146 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:29.216250 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:58:29.220454 (+ 4204us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.96.67:34851: BlockingRecv error: recv got EOF from 127.9.96.67:34851 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
Aug 06 04:58:29 dist-test-slave-x4qh krb5kdc[22124](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.96.65@KRBTEST.COM for oryx/127.9.96.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 04:58:29 dist-test-slave-x4qh krb5kdc[22124](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.96.65@KRBTEST.COM for oryx/127.9.96.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 04:58:29.288205 23556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:58:29.279681 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:44517 (local address 127.9.96.65:37729)
0806 04:58:29.280334 (+ 653us) negotiation.cc:107] Waiting for socket to connect
0806 04:58:29.280350 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 04:58:29.280510 (+ 160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:58:29.280887 (+ 377us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:58:29.280892 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:58:29.281288 (+ 396us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:58:29.281438 (+ 150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:58:29.281447 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:58:29.282491 (+ 1044us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:58:29.282499 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:58:29.283501 (+ 1002us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:58:29.283513 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:58:29.284241 (+ 728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:58:29.284246 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:58:29.284446 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:58:29.285005 (+ 559us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:58:29.285021 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:58:29.287876 (+ 2855us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.96.126:44517: Server oryx/127.9.96.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":536,"thread_start_us":417,"threads_started":1}
W20260806 04:58:29.288478 23150 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 04:58:29.279570 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.96.65:37729 (local address 127.9.96.126:44517)
0806 04:58:29.279896 (+ 326us) server_negotiation.cc:206] Beginning negotiation
0806 04:58:29.279904 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 04:58:29.280441 (+ 537us) server_negotiation.cc:407] Connection header received
0806 04:58:29.280638 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:58:29.280643 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:58:29.280718 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:58:29.280831 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:58:29.281588 (+ 757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:29.282356 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:29.283646 (+ 1290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:29.284072 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:29.284227 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:58:29.288175 (+ 3948us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.96.65:37729: BlockingRecv error: recv got EOF from 127.9.96.65:37729 (error 108)
Metrics: {"server-negotiator.queue_time_us":145}
Aug 06 04:58:29 dist-test-slave-x4qh krb5kdc[22124](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.96.66@KRBTEST.COM for oryx/127.9.96.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 04:58:29 dist-test-slave-x4qh krb5kdc[22124](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.96.66@KRBTEST.COM for oryx/127.9.96.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 04:58:29.744249 23557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:58:29.736121 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:44517 (local address 127.9.96.66:56665)
0806 04:58:29.736634 (+ 513us) negotiation.cc:107] Waiting for socket to connect
0806 04:58:29.736650 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 04:58:29.736811 (+ 161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:58:29.737148 (+ 337us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:58:29.737152 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:58:29.737572 (+ 420us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:58:29.737721 (+ 149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:58:29.737729 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:58:29.738731 (+ 1002us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:58:29.738737 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:58:29.739739 (+ 1002us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:58:29.739752 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:58:29.740444 (+ 692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:58:29.740449 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:58:29.740651 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:58:29.741220 (+ 569us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:58:29.741237 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:58:29.744030 (+ 2793us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.96.126:44517: Server oryx/127.9.96.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":392,"thread_start_us":295,"threads_started":1}
W20260806 04:58:29.744457 23150 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 04:58:29.736209 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.96.66:56665 (local address 127.9.96.126:44517)
0806 04:58:29.736419 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0806 04:58:29.736426 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 04:58:29.736741 (+ 315us) server_negotiation.cc:407] Connection header received
0806 04:58:29.736942 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:58:29.736947 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:58:29.737018 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:58:29.737116 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:58:29.737888 (+ 772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:29.738599 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:29.739885 (+ 1286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:29.740295 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:29.740421 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:58:29.744196 (+ 3775us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.96.66:56665: BlockingRecv error: recv got EOF from 127.9.96.66:56665 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
Aug 06 04:58:30 dist-test-slave-x4qh krb5kdc[22124](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.96.67@KRBTEST.COM for oryx/127.9.96.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 04:58:30 dist-test-slave-x4qh krb5kdc[22124](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.96.67@KRBTEST.COM for oryx/127.9.96.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 04:58:30.229791 23558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:58:30.221580 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:44517 (local address 127.9.96.67:33609)
0806 04:58:30.222093 (+ 513us) negotiation.cc:107] Waiting for socket to connect
0806 04:58:30.222110 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 04:58:30.222217 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:58:30.222528 (+ 311us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:58:30.222532 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:58:30.222915 (+ 383us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:58:30.223067 (+ 152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:58:30.223075 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:58:30.224252 (+ 1177us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:58:30.224257 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:58:30.225228 (+ 971us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:58:30.225239 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:58:30.225881 (+ 642us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:58:30.225885 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:58:30.226011 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:58:30.226580 (+ 569us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:58:30.226596 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:58:30.229465 (+ 2869us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.96.126:44517: Server oryx/127.9.96.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":391,"thread_start_us":291,"threads_started":1}
W20260806 04:58:30.229914 23150 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 04:58:30.221709 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.96.67:33609 (local address 127.9.96.126:44517)
0806 04:58:30.221878 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0806 04:58:30.221883 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 04:58:30.222183 (+ 300us) server_negotiation.cc:407] Connection header received
0806 04:58:30.222312 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:58:30.222315 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:58:30.222397 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:58:30.222482 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:58:30.223190 (+ 708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:30.224108 (+ 918us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:30.225351 (+ 1243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:30.225766 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:30.225853 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:58:30.229654 (+ 3801us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.96.67:33609: BlockingRecv error: recv got EOF from 127.9.96.67:33609 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
Aug 06 04:58:30 dist-test-slave-x4qh krb5kdc[22124](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.96.65@KRBTEST.COM for oryx/127.9.96.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 04:58:30 dist-test-slave-x4qh krb5kdc[22124](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.96.65@KRBTEST.COM for oryx/127.9.96.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 04:58:30.297674 23559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:58:30.289183 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:44517 (local address 127.9.96.65:55831)
0806 04:58:30.289791 (+ 608us) negotiation.cc:107] Waiting for socket to connect
0806 04:58:30.289808 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 04:58:30.289978 (+ 170us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:58:30.290313 (+ 335us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:58:30.290318 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:58:30.290777 (+ 459us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:58:30.290947 (+ 170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:58:30.290957 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:58:30.292066 (+ 1109us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:58:30.292074 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:58:30.292997 (+ 923us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:58:30.293008 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:58:30.293778 (+ 770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:58:30.293783 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:58:30.293968 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:58:30.294610 (+ 642us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:58:30.294627 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:58:30.297338 (+ 2711us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.96.126:44517: Server oryx/127.9.96.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":507,"thread_start_us":352,"threads_started":1}
W20260806 04:58:30.297757 23150 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 04:58:30.289225 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.96.65:55831 (local address 127.9.96.126:44517)
0806 04:58:30.289411 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0806 04:58:30.289419 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 04:58:30.289909 (+ 490us) server_negotiation.cc:407] Connection header received
0806 04:58:30.290103 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:58:30.290108 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:58:30.290176 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:58:30.290279 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:58:30.291106 (+ 827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:30.291928 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:30.293140 (+ 1212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:30.293608 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:30.293734 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:58:30.297547 (+ 3813us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.96.65:55831: BlockingRecv error: recv got EOF from 127.9.96.65:55831 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
Aug 06 04:58:30 dist-test-slave-x4qh krb5kdc[22124](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.96.66@KRBTEST.COM for oryx/127.9.96.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 04:58:30 dist-test-slave-x4qh krb5kdc[22124](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.96.66@KRBTEST.COM for oryx/127.9.96.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 04:58:30.754127 23561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:58:30.745255 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:44517 (local address 127.9.96.66:53235)
0806 04:58:30.745840 (+ 585us) negotiation.cc:107] Waiting for socket to connect
0806 04:58:30.745855 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 04:58:30.746009 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:58:30.746370 (+ 361us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:58:30.746375 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:58:30.746816 (+ 441us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:58:30.746975 (+ 159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:58:30.746983 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:58:30.748069 (+ 1086us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:58:30.748078 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:58:30.749137 (+ 1059us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:58:30.749150 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:58:30.749823 (+ 673us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:58:30.749829 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:58:30.750006 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:58:30.750786 (+ 780us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:58:30.750805 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:58:30.753838 (+ 3033us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.96.126:44517: Server oryx/127.9.96.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":475,"thread_start_us":289,"threads_started":1}
W20260806 04:58:30.754432 23150 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 04:58:30.745464 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.96.66:53235 (local address 127.9.96.126:44517)
0806 04:58:30.745671 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0806 04:58:30.745678 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 04:58:30.745939 (+ 261us) server_negotiation.cc:407] Connection header received
0806 04:58:30.746138 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:58:30.746142 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:58:30.746215 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:58:30.746324 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:58:30.747117 (+ 793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:30.747931 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:30.749271 (+ 1340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:30.749682 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:30.749812 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:58:30.754093 (+ 4281us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.96.66:53235: BlockingRecv error: recv got EOF from 127.9.96.66:53235 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
Aug 06 04:58:31 dist-test-slave-x4qh krb5kdc[22124](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.96.67@KRBTEST.COM for oryx/127.9.96.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 04:58:31 dist-test-slave-x4qh krb5kdc[22124](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.96.67@KRBTEST.COM for oryx/127.9.96.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 04:58:31.238863 23563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:58:31.230783 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:44517 (local address 127.9.96.67:59985)
0806 04:58:31.231314 (+ 531us) negotiation.cc:107] Waiting for socket to connect
0806 04:58:31.231330 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 04:58:31.231519 (+ 189us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:58:31.231860 (+ 341us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:58:31.231865 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:58:31.232297 (+ 432us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:58:31.232467 (+ 170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:58:31.232476 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:58:31.233530 (+ 1054us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:58:31.233534 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:58:31.234440 (+ 906us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:58:31.234450 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:58:31.235084 (+ 634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:58:31.235088 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:58:31.235202 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:58:31.235798 (+ 596us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:58:31.235815 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:58:31.238596 (+ 2781us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.96.126:44517: Server oryx/127.9.96.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":434,"thread_start_us":264,"threads_started":1}
W20260806 04:58:31.239037 23150 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 04:58:31.230874 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.96.67:59985 (local address 127.9.96.126:44517)
0806 04:58:31.231105 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0806 04:58:31.231113 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 04:58:31.231436 (+ 323us) server_negotiation.cc:407] Connection header received
0806 04:58:31.231646 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:58:31.231651 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:58:31.231721 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:58:31.231827 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:58:31.232649 (+ 822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:31.233397 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:31.234559 (+ 1162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:31.234968 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:31.235059 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:58:31.238772 (+ 3713us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.96.67:59985: BlockingRecv error: recv got EOF from 127.9.96.67:59985 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
Aug 06 04:58:31 dist-test-slave-x4qh krb5kdc[22124](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.96.65@KRBTEST.COM for oryx/127.9.96.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 04:58:31 dist-test-slave-x4qh krb5kdc[22124](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.96.65@KRBTEST.COM for oryx/127.9.96.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 04:58:31.307104 23564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:58:31.298689 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:44517 (local address 127.9.96.65:37357)
0806 04:58:31.299446 (+ 757us) negotiation.cc:107] Waiting for socket to connect
0806 04:58:31.299461 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 04:58:31.299607 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:58:31.299957 (+ 350us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:58:31.299962 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:58:31.300386 (+ 424us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:58:31.300563 (+ 177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:58:31.300572 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:58:31.301630 (+ 1058us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:58:31.301635 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:58:31.302553 (+ 918us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:58:31.302564 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:58:31.303303 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:58:31.303310 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:58:31.303529 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:58:31.304071 (+ 542us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:58:31.304088 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:58:31.306798 (+ 2710us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.96.126:44517: Server oryx/127.9.96.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":622,"thread_start_us":482,"threads_started":1}
W20260806 04:58:31.307245 23150 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 04:58:31.298715 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.96.65:37357 (local address 127.9.96.126:44517)
0806 04:58:31.298962 (+ 247us) server_negotiation.cc:206] Beginning negotiation
0806 04:58:31.298968 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 04:58:31.299549 (+ 581us) server_negotiation.cc:407] Connection header received
0806 04:58:31.299737 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:58:31.299742 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:58:31.299816 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:58:31.299926 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:58:31.300713 (+ 787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:31.301482 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:31.302700 (+ 1218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:31.303134 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:31.303265 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:58:31.306999 (+ 3734us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.96.65:37357: BlockingRecv error: recv got EOF from 127.9.96.65:37357 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
Aug 06 04:58:31 dist-test-slave-x4qh krb5kdc[22124](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.96.66@KRBTEST.COM for oryx/127.9.96.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 04:58:31 dist-test-slave-x4qh krb5kdc[22124](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.96.66@KRBTEST.COM for oryx/127.9.96.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 04:58:31.763154 23565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:58:31.755132 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:44517 (local address 127.9.96.66:44767)
0806 04:58:31.755676 (+ 544us) negotiation.cc:107] Waiting for socket to connect
0806 04:58:31.755691 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 04:58:31.755842 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:58:31.756189 (+ 347us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:58:31.756194 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:58:31.756616 (+ 422us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:58:31.756795 (+ 179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:58:31.756805 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:58:31.757780 (+ 975us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:58:31.757786 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:58:31.758656 (+ 870us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:58:31.758667 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:58:31.759416 (+ 749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:58:31.759423 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:58:31.759606 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:58:31.760150 (+ 544us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:58:31.760167 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:58:31.762844 (+ 2677us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.96.126:44517: Server oryx/127.9.96.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":452,"thread_start_us":340,"threads_started":1}
W20260806 04:58:31.763341 23150 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 04:58:31.755132 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.96.66:44767 (local address 127.9.96.126:44517)
0806 04:58:31.755344 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0806 04:58:31.755352 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 04:58:31.755783 (+ 431us) server_negotiation.cc:407] Connection header received
0806 04:58:31.755996 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:58:31.756000 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:58:31.756059 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:58:31.756156 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:58:31.756955 (+ 799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:31.757643 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:31.758809 (+ 1166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:31.759205 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:31.759351 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:58:31.763092 (+ 3741us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.96.66:44767: BlockingRecv error: recv got EOF from 127.9.96.66:44767 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
Aug 06 04:58:32 dist-test-slave-x4qh krb5kdc[22124](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.96.67@KRBTEST.COM for oryx/127.9.96.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 04:58:32 dist-test-slave-x4qh krb5kdc[22124](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.96.67@KRBTEST.COM for oryx/127.9.96.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 04:58:32.251293 23567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:58:32.239912 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:44517 (local address 127.9.96.67:34029)
0806 04:58:32.240475 (+ 563us) negotiation.cc:107] Waiting for socket to connect
0806 04:58:32.240490 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 04:58:32.240642 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:58:32.240971 (+ 329us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:58:32.240975 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:58:32.241410 (+ 435us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:58:32.241557 (+ 147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:58:32.241565 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:58:32.242628 (+ 1063us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:58:32.242633 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:58:32.243537 (+ 904us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:58:32.243548 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:58:32.244325 (+ 777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:58:32.244330 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:58:32.244516 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:58:32.247959 (+ 3443us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:58:32.247975 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:58:32.251056 (+ 3081us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.96.126:44517: Server oryx/127.9.96.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":477,"thread_start_us":334,"threads_started":1}
W20260806 04:58:32.251519 23150 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 04:58:32.239982 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.96.67:34029 (local address 127.9.96.126:44517)
0806 04:58:32.240217 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0806 04:58:32.240224 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 04:58:32.240578 (+ 354us) server_negotiation.cc:407] Connection header received
0806 04:58:32.240770 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:58:32.240774 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:58:32.240841 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:58:32.240941 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:58:32.241706 (+ 765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:32.242486 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:32.243683 (+ 1197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:32.244127 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:32.244281 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:58:32.251265 (+ 6984us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.96.67:34029: BlockingRecv error: recv got EOF from 127.9.96.67:34029 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
Aug 06 04:58:32 dist-test-slave-x4qh krb5kdc[22124](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.96.65@KRBTEST.COM for oryx/127.9.96.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 04:58:32 dist-test-slave-x4qh krb5kdc[22124](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.96.65@KRBTEST.COM for oryx/127.9.96.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 04:58:32.316157 23569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:58:32.308258 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:44517 (local address 127.9.96.65:37343)
0806 04:58:32.308932 (+ 674us) negotiation.cc:107] Waiting for socket to connect
0806 04:58:32.308948 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 04:58:32.309065 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:58:32.309370 (+ 305us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:58:32.309373 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:58:32.309765 (+ 392us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:58:32.309957 (+ 192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:58:32.309966 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:58:32.310881 (+ 915us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:58:32.310885 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:58:32.311842 (+ 957us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:58:32.311854 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:58:32.312513 (+ 659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:58:32.312516 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:58:32.312627 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:58:32.313175 (+ 548us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:58:32.313191 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:58:32.315891 (+ 2700us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.96.126:44517: Server oryx/127.9.96.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":559,"thread_start_us":325,"threads_started":1}
W20260806 04:58:32.316340 23150 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 04:58:32.308326 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.96.65:37343 (local address 127.9.96.126:44517)
0806 04:58:32.308583 (+ 257us) server_negotiation.cc:206] Beginning negotiation
0806 04:58:32.308591 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 04:58:32.309029 (+ 438us) server_negotiation.cc:407] Connection header received
0806 04:58:32.309167 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:58:32.309170 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:58:32.309232 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:58:32.309310 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:58:32.310085 (+ 775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:32.310759 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:32.311964 (+ 1205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:32.312376 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:32.312479 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:58:32.316111 (+ 3632us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.96.65:37343: BlockingRecv error: recv got EOF from 127.9.96.65:37343 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
Aug 06 04:58:32 dist-test-slave-x4qh krb5kdc[22124](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.96.66@KRBTEST.COM for oryx/127.9.96.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 04:58:32 dist-test-slave-x4qh krb5kdc[22124](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.96.66@KRBTEST.COM for oryx/127.9.96.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 04:58:32.774183 23571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:58:32.764090 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:44517 (local address 127.9.96.66:38661)
0806 04:58:32.764750 (+ 660us) negotiation.cc:107] Waiting for socket to connect
0806 04:58:32.764767 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 04:58:32.764880 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:58:32.765288 (+ 408us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:58:32.765291 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:58:32.765711 (+ 420us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:58:32.765883 (+ 172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:58:32.765892 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:58:32.767599 (+ 1707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:58:32.767603 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:58:32.768520 (+ 917us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:58:32.768532 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:58:32.769708 (+ 1176us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:58:32.769713 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:58:32.769823 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:58:32.770487 (+ 664us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:58:32.770518 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:58:32.773936 (+ 3418us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.96.126:44517: Server oryx/127.9.96.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":542,"thread_start_us":325,"threads_started":1}
W20260806 04:58:32.774567 23150 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 04:58:32.764213 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.96.66:38661 (local address 127.9.96.126:44517)
0806 04:58:32.764498 (+ 285us) server_negotiation.cc:206] Beginning negotiation
0806 04:58:32.764507 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 04:58:32.764854 (+ 347us) server_negotiation.cc:407] Connection header received
0806 04:58:32.765019 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:58:32.765027 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:58:32.765133 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:58:32.765259 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:58:32.766050 (+ 791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:32.767462 (+ 1412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:32.768670 (+ 1208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:32.769559 (+ 889us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:32.769703 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:58:32.774216 (+ 4513us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.96.66:38661: BlockingRecv error: recv got EOF from 127.9.96.66:38661 (error 108)
Metrics: {"server-negotiator.queue_time_us":111}
Aug 06 04:58:33 dist-test-slave-x4qh krb5kdc[22124](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.96.67@KRBTEST.COM for oryx/127.9.96.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 04:58:33 dist-test-slave-x4qh krb5kdc[22124](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.96.67@KRBTEST.COM for oryx/127.9.96.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 04:58:33.260526 23573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:58:33.252148 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:44517 (local address 127.9.96.67:58359)
0806 04:58:33.252728 (+ 580us) negotiation.cc:107] Waiting for socket to connect
0806 04:58:33.252750 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0806 04:58:33.252912 (+ 162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:58:33.253294 (+ 382us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:58:33.253306 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:58:33.253722 (+ 416us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:58:33.253956 (+ 234us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:58:33.253966 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:58:33.254959 (+ 993us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:58:33.254963 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:58:33.255878 (+ 915us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:58:33.255889 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:58:33.256655 (+ 766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:58:33.256659 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:58:33.256771 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:58:33.257421 (+ 650us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:58:33.257438 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:58:33.260151 (+ 2713us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.96.126:44517: Server oryx/127.9.96.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":452,"thread_start_us":247,"threads_started":1}
W20260806 04:58:33.260560 23150 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 04:58:33.252224 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.96.67:58359 (local address 127.9.96.126:44517)
0806 04:58:33.252448 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0806 04:58:33.252458 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 04:58:33.252860 (+ 402us) server_negotiation.cc:407] Connection header received
0806 04:58:33.253025 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:58:33.253028 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:58:33.253113 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:58:33.253215 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:58:33.254082 (+ 867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:33.254842 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:33.256001 (+ 1159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:33.256527 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:33.256620 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:58:33.260337 (+ 3717us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.96.67:58359: BlockingRecv error: recv got EOF from 127.9.96.67:58359 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
Aug 06 04:58:33 dist-test-slave-x4qh krb5kdc[22124](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.96.65@KRBTEST.COM for oryx/127.9.96.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 04:58:33 dist-test-slave-x4qh krb5kdc[22124](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.96.65@KRBTEST.COM for oryx/127.9.96.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 04:58:33.326131 23574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:58:33.317188 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:44517 (local address 127.9.96.65:36079)
0806 04:58:33.317823 (+ 635us) negotiation.cc:107] Waiting for socket to connect
0806 04:58:33.317839 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 04:58:33.317996 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:58:33.318343 (+ 347us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:58:33.318349 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:58:33.318804 (+ 455us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:58:33.318953 (+ 149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:58:33.318962 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:58:33.320168 (+ 1206us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:58:33.320173 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:58:33.321267 (+ 1094us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:58:33.321281 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:58:33.322124 (+ 843us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:58:33.322129 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:58:33.322306 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:58:33.322960 (+ 654us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:58:33.322979 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:58:33.325900 (+ 2921us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.96.126:44517: Server oryx/127.9.96.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":501,"thread_start_us":377,"threads_started":1}
W20260806 04:58:33.326308 23150 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 04:58:33.317399 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.96.65:36079 (local address 127.9.96.126:44517)
0806 04:58:33.317700 (+ 301us) server_negotiation.cc:206] Beginning negotiation
0806 04:58:33.317707 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 04:58:33.317923 (+ 216us) server_negotiation.cc:407] Connection header received
0806 04:58:33.318121 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:58:33.318126 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:58:33.318195 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:58:33.318298 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:58:33.319099 (+ 801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:33.320004 (+ 905us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:33.321420 (+ 1416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:33.321939 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:33.322080 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:58:33.326079 (+ 3999us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.96.65:36079: BlockingRecv error: recv got EOF from 127.9.96.65:36079 (error 108)
Metrics: {"server-negotiator.queue_time_us":106}
Aug 06 04:58:33 dist-test-slave-x4qh krb5kdc[22124](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.96.66@KRBTEST.COM for oryx/127.9.96.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 04:58:33 dist-test-slave-x4qh krb5kdc[22124](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.96.66@KRBTEST.COM for oryx/127.9.96.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 04:58:33.784282 23575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:58:33.775276 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:44517 (local address 127.9.96.66:52113)
0806 04:58:33.775880 (+ 604us) negotiation.cc:107] Waiting for socket to connect
0806 04:58:33.775896 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 04:58:33.776074 (+ 178us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:58:33.776470 (+ 396us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:58:33.776476 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:58:33.776950 (+ 474us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:58:33.777103 (+ 153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:58:33.777111 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:58:33.778333 (+ 1222us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:58:33.778340 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:58:33.779349 (+ 1009us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:58:33.779362 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:58:33.780370 (+ 1008us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:58:33.780375 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:58:33.780545 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:58:33.781173 (+ 628us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:58:33.781190 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:58:33.783941 (+ 2751us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.96.126:44517: Server oryx/127.9.96.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":456,"thread_start_us":329,"threads_started":1}
W20260806 04:58:33.784401 23150 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 04:58:33.775338 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.96.66:52113 (local address 127.9.96.126:44517)
0806 04:58:33.775542 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0806 04:58:33.775547 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 04:58:33.776000 (+ 453us) server_negotiation.cc:407] Connection header received
0806 04:58:33.776200 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:58:33.776205 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:58:33.776309 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:58:33.776417 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:58:33.777256 (+ 839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:33.778161 (+ 905us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:33.779547 (+ 1386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:33.780192 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:33.780339 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:58:33.784135 (+ 3796us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.96.66:52113: BlockingRecv error: recv got EOF from 127.9.96.66:52113 (error 108)
Metrics: {"server-negotiator.queue_time_us":92}
Aug 06 04:58:34 dist-test-slave-x4qh krb5kdc[22124](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.96.67@KRBTEST.COM for oryx/127.9.96.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 04:58:34 dist-test-slave-x4qh krb5kdc[22124](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.96.67@KRBTEST.COM for oryx/127.9.96.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 04:58:34.270165 23576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:58:34.261527 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:44517 (local address 127.9.96.67:40085)
0806 04:58:34.262048 (+ 521us) negotiation.cc:107] Waiting for socket to connect
0806 04:58:34.262064 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 04:58:34.262222 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:58:34.262565 (+ 343us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:58:34.262570 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:58:34.263008 (+ 438us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:58:34.263161 (+ 153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:58:34.263170 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:58:34.264235 (+ 1065us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:58:34.264241 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:58:34.265203 (+ 962us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:58:34.265219 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:58:34.266224 (+ 1005us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:58:34.266231 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:58:34.266396 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:58:34.267121 (+ 725us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:58:34.267138 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:58:34.269946 (+ 2808us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.96.126:44517: Server oryx/127.9.96.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":408,"thread_start_us":262,"threads_started":1}
W20260806 04:58:34.270370 23150 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 04:58:34.261614 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.96.67:40085 (local address 127.9.96.126:44517)
0806 04:58:34.261852 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0806 04:58:34.261860 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 04:58:34.262155 (+ 295us) server_negotiation.cc:407] Connection header received
0806 04:58:34.262349 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:58:34.262354 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:58:34.262427 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:58:34.262533 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:58:34.263307 (+ 774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:34.264086 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:34.265379 (+ 1293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:34.266026 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:34.266158 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:58:34.270129 (+ 3971us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.96.67:40085: BlockingRecv error: recv got EOF from 127.9.96.67:40085 (error 108)
Metrics: {"server-negotiator.queue_time_us":83}
Aug 06 04:58:34 dist-test-slave-x4qh krb5kdc[22124](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.96.65@KRBTEST.COM for oryx/127.9.96.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 04:58:34 dist-test-slave-x4qh krb5kdc[22124](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.96.65@KRBTEST.COM for oryx/127.9.96.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 04:58:34.335175 23578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:58:34.327066 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:44517 (local address 127.9.96.65:50817)
0806 04:58:34.327628 (+ 562us) negotiation.cc:107] Waiting for socket to connect
0806 04:58:34.327645 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 04:58:34.327759 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:58:34.328082 (+ 323us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:58:34.328086 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:58:34.328469 (+ 383us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:58:34.328639 (+ 170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:58:34.328647 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:58:34.329601 (+ 954us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:58:34.329604 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:58:34.330590 (+ 986us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:58:34.330603 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:58:34.331263 (+ 660us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:58:34.331267 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:58:34.331471 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:58:34.332104 (+ 633us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:58:34.332121 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:58:34.334912 (+ 2791us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.96.126:44517: Server oryx/127.9.96.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":443,"thread_start_us":346,"threads_started":1}
W20260806 04:58:34.335314 23150 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 04:58:34.327097 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.96.65:50817 (local address 127.9.96.126:44517)
0806 04:58:34.327290 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0806 04:58:34.327295 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 04:58:34.327719 (+ 424us) server_negotiation.cc:407] Connection header received
0806 04:58:34.327875 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:58:34.327878 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:58:34.327940 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:58:34.328021 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:58:34.328763 (+ 742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:34.329487 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:34.330725 (+ 1238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:34.331131 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:34.331223 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:58:34.335108 (+ 3885us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.96.65:50817: BlockingRecv error: recv got EOF from 127.9.96.65:50817 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
Aug 06 04:58:34 dist-test-slave-x4qh krb5kdc[22124](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.96.66@KRBTEST.COM for oryx/127.9.96.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 04:58:34 dist-test-slave-x4qh krb5kdc[22124](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.96.66@KRBTEST.COM for oryx/127.9.96.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 04:58:34.794104 23579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:58:34.785486 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:44517 (local address 127.9.96.66:47909)
0806 04:58:34.786123 (+ 637us) negotiation.cc:107] Waiting for socket to connect
0806 04:58:34.786138 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 04:58:34.786256 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:58:34.786644 (+ 388us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:58:34.786649 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:58:34.787097 (+ 448us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:58:34.787270 (+ 173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:58:34.787280 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:58:34.788460 (+ 1180us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:58:34.788467 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:58:34.789453 (+ 986us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:58:34.789467 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:58:34.790199 (+ 732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:58:34.790205 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:58:34.790386 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:58:34.790941 (+ 555us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:58:34.790959 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:58:34.793853 (+ 2894us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.96.126:44517: Server oryx/127.9.96.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":532,"thread_start_us":385,"threads_started":1}
W20260806 04:58:34.794286 23150 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 04:58:34.785607 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.96.66:47909 (local address 127.9.96.126:44517)
0806 04:58:34.785817 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0806 04:58:34.785824 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 04:58:34.786229 (+ 405us) server_negotiation.cc:407] Connection header received
0806 04:58:34.786374 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:58:34.786381 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:58:34.786465 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:58:34.786583 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:58:34.787431 (+ 848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:34.788321 (+ 890us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:34.789623 (+ 1302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:34.790032 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:34.790161 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:58:34.794064 (+ 3903us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.96.66:47909: BlockingRecv error: recv got EOF from 127.9.96.66:47909 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
Aug 06 04:58:35 dist-test-slave-x4qh krb5kdc[22124](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.96.67@KRBTEST.COM for oryx/127.9.96.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 04:58:35 dist-test-slave-x4qh krb5kdc[22124](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.96.67@KRBTEST.COM for oryx/127.9.96.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 04:58:35.279745 23580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:58:35.271331 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:44517 (local address 127.9.96.67:35251)
0806 04:58:35.272063 (+ 732us) negotiation.cc:107] Waiting for socket to connect
0806 04:58:35.272086 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0806 04:58:35.272219 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:58:35.272560 (+ 341us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:58:35.272563 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:58:35.272944 (+ 381us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:58:35.273106 (+ 162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:58:35.273115 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:58:35.274133 (+ 1018us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:58:35.274137 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:58:35.275066 (+ 929us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:58:35.275077 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:58:35.275926 (+ 849us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:58:35.275931 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:58:35.276049 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:58:35.276629 (+ 580us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:58:35.276645 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:58:35.279310 (+ 2665us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.96.126:44517: Server oryx/127.9.96.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":587,"thread_start_us":413,"threads_started":1}
W20260806 04:58:35.279816 23150 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 04:58:35.271360 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.96.67:35251 (local address 127.9.96.126:44517)
0806 04:58:35.271598 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0806 04:58:35.271609 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 04:58:35.272183 (+ 574us) server_negotiation.cc:407] Connection header received
0806 04:58:35.272322 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:58:35.272325 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:58:35.272415 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:58:35.272507 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:58:35.273239 (+ 732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:35.274017 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:35.275236 (+ 1219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:35.275791 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:35.275899 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:58:35.279631 (+ 3732us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.96.67:35251: BlockingRecv error: recv got EOF from 127.9.96.67:35251 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
Aug 06 04:58:35 dist-test-slave-x4qh krb5kdc[22124](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.96.65@KRBTEST.COM for oryx/127.9.96.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 04:58:35 dist-test-slave-x4qh krb5kdc[22124](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.96.65@KRBTEST.COM for oryx/127.9.96.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 04:58:35.344348 23582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:58:35.336133 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:44517 (local address 127.9.96.65:50359)
0806 04:58:35.336609 (+ 476us) negotiation.cc:107] Waiting for socket to connect
0806 04:58:35.336627 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 04:58:35.336749 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:58:35.337059 (+ 310us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:58:35.337063 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:58:35.337430 (+ 367us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:58:35.337585 (+ 155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:58:35.337593 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:58:35.338584 (+ 991us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:58:35.338587 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:58:35.339650 (+ 1063us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:58:35.339662 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:58:35.340360 (+ 698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:58:35.340364 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:58:35.340488 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:58:35.341034 (+ 546us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:58:35.341051 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:58:35.344100 (+ 3049us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.96.126:44517: Server oryx/127.9.96.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":368,"thread_start_us":262,"threads_started":1}
W20260806 04:58:35.344537 23150 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 04:58:35.336306 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.96.65:50359 (local address 127.9.96.126:44517)
0806 04:58:35.336525 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0806 04:58:35.336531 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 04:58:35.336705 (+ 174us) server_negotiation.cc:407] Connection header received
0806 04:58:35.336850 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:58:35.336853 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:58:35.336928 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:58:35.337008 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:58:35.337709 (+ 701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:35.338450 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:35.339776 (+ 1326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:35.340223 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:35.340320 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:58:35.344270 (+ 3950us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.96.65:50359: BlockingRecv error: recv got EOF from 127.9.96.65:50359 (error 108)
Metrics: {"server-negotiator.queue_time_us":81}
Aug 06 04:58:35 dist-test-slave-x4qh krb5kdc[22124](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.96.66@KRBTEST.COM for oryx/127.9.96.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 04:58:35 dist-test-slave-x4qh krb5kdc[22124](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.96.66@KRBTEST.COM for oryx/127.9.96.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 04:58:35.807024 23583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:58:35.795048 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:44517 (local address 127.9.96.66:60693)
0806 04:58:35.795658 (+ 610us) negotiation.cc:107] Waiting for socket to connect
0806 04:58:35.795674 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 04:58:35.795781 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:58:35.796082 (+ 301us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:58:35.796085 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:58:35.796475 (+ 390us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:58:35.796698 (+ 223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:58:35.796713 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:58:35.798125 (+ 1412us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:58:35.798132 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:58:35.799063 (+ 931us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:58:35.799075 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:58:35.800007 (+ 932us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:58:35.800012 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:58:35.800137 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:58:35.803928 (+ 3791us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:58:35.803947 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:58:35.806660 (+ 2713us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.96.126:44517: Server oryx/127.9.96.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":494,"thread_start_us":361,"threads_started":1}
W20260806 04:58:35.807127 23150 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 04:58:35.795269 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.96.66:60693 (local address 127.9.96.126:44517)
0806 04:58:35.795473 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0806 04:58:35.795478 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 04:58:35.795746 (+ 268us) server_negotiation.cc:407] Connection header received
0806 04:58:35.795887 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:58:35.795890 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:58:35.795950 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:58:35.796034 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:58:35.796898 (+ 864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:35.797997 (+ 1099us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:35.799214 (+ 1217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:35.799677 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:35.799811 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:58:35.806875 (+ 7064us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.96.66:60693: BlockingRecv error: recv got EOF from 127.9.96.66:60693 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
Aug 06 04:58:36 dist-test-slave-x4qh krb5kdc[22124](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.96.67@KRBTEST.COM for oryx/127.9.96.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 04:58:36 dist-test-slave-x4qh krb5kdc[22124](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.96.67@KRBTEST.COM for oryx/127.9.96.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 04:58:36.289430 23584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:58:36.280854 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:44517 (local address 127.9.96.67:50385)
0806 04:58:36.281434 (+ 580us) negotiation.cc:107] Waiting for socket to connect
0806 04:58:36.281452 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 04:58:36.281609 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:58:36.281935 (+ 326us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:58:36.281938 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:58:36.282476 (+ 538us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:58:36.282675 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:58:36.282687 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:58:36.284103 (+ 1416us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:58:36.284107 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:58:36.285040 (+ 933us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:58:36.285052 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:58:36.285675 (+ 623us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:58:36.285679 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:58:36.285813 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:58:36.286374 (+ 561us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:58:36.286392 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:58:36.289208 (+ 2816us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.96.126:44517: Server oryx/127.9.96.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":459,"thread_start_us":334,"threads_started":1}
W20260806 04:58:36.289625 23150 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 04:58:36.280988 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.96.67:50385 (local address 127.9.96.126:44517)
0806 04:58:36.281154 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0806 04:58:36.281158 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 04:58:36.281528 (+ 370us) server_negotiation.cc:407] Connection header received
0806 04:58:36.281717 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:58:36.281720 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:58:36.281792 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:58:36.281879 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:58:36.282833 (+ 954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:36.283970 (+ 1137us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:36.285156 (+ 1186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:36.285549 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:36.285639 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:58:36.289403 (+ 3764us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.96.67:50385: BlockingRecv error: recv got EOF from 127.9.96.67:50385 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
Aug 06 04:58:36 dist-test-slave-x4qh krb5kdc[22124](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.96.65@KRBTEST.COM for oryx/127.9.96.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 04:58:36 dist-test-slave-x4qh krb5kdc[22124](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.96.65@KRBTEST.COM for oryx/127.9.96.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 04:58:36.353367 23585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:58:36.345219 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:44517 (local address 127.9.96.65:43347)
0806 04:58:36.345734 (+ 515us) negotiation.cc:107] Waiting for socket to connect
0806 04:58:36.345750 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 04:58:36.345868 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:58:36.346159 (+ 291us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:58:36.346162 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:58:36.346554 (+ 392us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:58:36.346728 (+ 174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:58:36.346736 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:58:36.347681 (+ 945us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:58:36.347685 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:58:36.348562 (+ 877us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:58:36.348572 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:58:36.349277 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:58:36.349281 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:58:36.349401 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:58:36.350049 (+ 648us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:58:36.350074 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:58:36.353006 (+ 2932us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.96.126:44517: Server oryx/127.9.96.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":410,"thread_start_us":288,"threads_started":1}
W20260806 04:58:36.353830 23150 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 04:58:36.345336 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.96.65:43347 (local address 127.9.96.126:44517)
0806 04:58:36.345537 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0806 04:58:36.345541 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 04:58:36.345816 (+ 275us) server_negotiation.cc:407] Connection header received
0806 04:58:36.345959 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:58:36.345963 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:58:36.346033 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:58:36.346115 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:58:36.346843 (+ 728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:36.347558 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:36.348676 (+ 1118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:36.349121 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:36.349209 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:58:36.353247 (+ 4038us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.96.65:43347: BlockingRecv error: recv got EOF from 127.9.96.65:43347 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
Aug 06 04:58:36 dist-test-slave-x4qh krb5kdc[22124](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.96.66@KRBTEST.COM for oryx/127.9.96.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 04:58:36 dist-test-slave-x4qh krb5kdc[22124](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.96.66@KRBTEST.COM for oryx/127.9.96.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 04:58:36.817025 23587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:58:36.808086 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:44517 (local address 127.9.96.66:54519)
0806 04:58:36.808706 (+ 620us) negotiation.cc:107] Waiting for socket to connect
0806 04:58:36.808721 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 04:58:36.808885 (+ 164us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:58:36.809222 (+ 337us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:58:36.809227 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:58:36.809679 (+ 452us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:58:36.809876 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:58:36.809885 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:58:36.810950 (+ 1065us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:58:36.810957 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:58:36.812109 (+ 1152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:58:36.812123 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:58:36.812905 (+ 782us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:58:36.812911 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:58:36.813098 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:58:36.813736 (+ 638us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:58:36.813754 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:58:36.816781 (+ 3027us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.96.126:44517: Server oryx/127.9.96.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":523,"thread_start_us":370,"threads_started":1}
W20260806 04:58:36.817310 23150 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 04:58:36.808197 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.96.66:54519 (local address 127.9.96.126:44517)
0806 04:58:36.808536 (+ 339us) server_negotiation.cc:206] Beginning negotiation
0806 04:58:36.808543 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 04:58:36.808811 (+ 268us) server_negotiation.cc:407] Connection header received
0806 04:58:36.809021 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:58:36.809025 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:58:36.809085 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:58:36.809188 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:58:36.810030 (+ 842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:36.810796 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:36.812251 (+ 1455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:36.812715 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:36.812856 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:58:36.817015 (+ 4159us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.96.66:54519: BlockingRecv error: recv got EOF from 127.9.96.66:54519 (error 108)
Metrics: {"server-negotiator.queue_time_us":181}
Aug 06 04:58:37 dist-test-slave-x4qh krb5kdc[22124](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.96.67@KRBTEST.COM for oryx/127.9.96.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 04:58:37 dist-test-slave-x4qh krb5kdc[22124](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.96.67@KRBTEST.COM for oryx/127.9.96.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 04:58:37.303412 23588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:58:37.290529 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:44517 (local address 127.9.96.67:51925)
0806 04:58:37.291082 (+ 553us) negotiation.cc:107] Waiting for socket to connect
0806 04:58:37.291099 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 04:58:37.291210 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:58:37.291660 (+ 450us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:58:37.291667 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:58:37.292099 (+ 432us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:58:37.292274 (+ 175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:58:37.292284 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:58:37.293327 (+ 1043us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:58:37.293332 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:58:37.294235 (+ 903us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:58:37.294246 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:58:37.294971 (+ 725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:58:37.294977 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:58:37.295156 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:58:37.300045 (+ 4889us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:58:37.300063 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:58:37.303020 (+ 2957us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.96.126:44517: Server oryx/127.9.96.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":446,"thread_start_us":320,"threads_started":1}
W20260806 04:58:37.303510 23150 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 04:58:37.290632 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.96.67:51925 (local address 127.9.96.126:44517)
0806 04:58:37.290846 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0806 04:58:37.290852 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 04:58:37.291182 (+ 330us) server_negotiation.cc:407] Connection header received
0806 04:58:37.291417 (+ 235us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:58:37.291426 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:58:37.291506 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:58:37.291614 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:58:37.292431 (+ 817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:37.293186 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:37.294383 (+ 1197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:37.294804 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:37.294950 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:58:37.303238 (+ 8288us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.96.67:51925: BlockingRecv error: recv got EOF from 127.9.96.67:51925 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
Aug 06 04:58:37 dist-test-slave-x4qh krb5kdc[22124](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.96.65@KRBTEST.COM for oryx/127.9.96.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 04:58:37 dist-test-slave-x4qh krb5kdc[22124](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.96.65@KRBTEST.COM for oryx/127.9.96.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 04:58:37.362648 23589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:58:37.354254 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:44517 (local address 127.9.96.65:50119)
0806 04:58:37.354763 (+ 509us) negotiation.cc:107] Waiting for socket to connect
0806 04:58:37.354779 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 04:58:37.354892 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:58:37.355230 (+ 338us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:58:37.355233 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:58:37.355711 (+ 478us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:58:37.355912 (+ 201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:58:37.355920 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:58:37.356941 (+ 1021us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:58:37.356945 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:58:37.357870 (+ 925us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:58:37.357881 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:58:37.358568 (+ 687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:58:37.358572 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:58:37.358703 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:58:37.359411 (+ 708us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:58:37.359435 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:58:37.362283 (+ 2848us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.96.126:44517: Server oryx/127.9.96.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":415,"thread_start_us":307,"threads_started":1}
W20260806 04:58:37.362730 23150 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 04:58:37.354313 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.96.65:50119 (local address 127.9.96.126:44517)
0806 04:58:37.354550 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0806 04:58:37.354559 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 04:58:37.354872 (+ 313us) server_negotiation.cc:407] Connection header received
0806 04:58:37.355019 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:58:37.355022 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:58:37.355096 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:58:37.355175 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:58:37.356039 (+ 864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:37.356757 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:37.357986 (+ 1229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:37.358444 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:37.358539 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:58:37.362489 (+ 3950us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.96.65:50119: BlockingRecv error: recv got EOF from 127.9.96.65:50119 (error 108)
Metrics: {"server-negotiator.queue_time_us":81}
Aug 06 04:58:37 dist-test-slave-x4qh krb5kdc[22124](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.96.66@KRBTEST.COM for oryx/127.9.96.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 04:58:37 dist-test-slave-x4qh krb5kdc[22124](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.96.66@KRBTEST.COM for oryx/127.9.96.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 04:58:37.825982 23590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:58:37.817959 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:44517 (local address 127.9.96.66:48371)
0806 04:58:37.818434 (+ 475us) negotiation.cc:107] Waiting for socket to connect
0806 04:58:37.818450 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 04:58:37.818561 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:58:37.818847 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:58:37.818850 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:58:37.819215 (+ 365us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:58:37.819399 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:58:37.819408 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:58:37.820449 (+ 1041us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:58:37.820454 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:58:37.821560 (+ 1106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:58:37.821573 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:58:37.822385 (+ 812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:58:37.822389 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:58:37.822524 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:58:37.823087 (+ 563us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:58:37.823103 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:58:37.825760 (+ 2657us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.96.126:44517: Server oryx/127.9.96.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":371,"thread_start_us":261,"threads_started":1}
W20260806 04:58:37.826146 23150 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 04:58:37.818052 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.96.66:48371 (local address 127.9.96.126:44517)
0806 04:58:37.818294 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0806 04:58:37.818301 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 04:58:37.818526 (+ 225us) server_negotiation.cc:407] Connection header received
0806 04:58:37.818653 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:58:37.818657 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:58:37.818716 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:58:37.818794 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:58:37.819514 (+ 720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:37.820284 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:37.821685 (+ 1401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:37.822251 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:37.822347 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:58:37.825938 (+ 3591us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.96.66:48371: BlockingRecv error: recv got EOF from 127.9.96.66:48371 (error 108)
Metrics: {"server-negotiator.queue_time_us":92}
Aug 06 04:58:38 dist-test-slave-x4qh krb5kdc[22124](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.96.67@KRBTEST.COM for oryx/127.9.96.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 04:58:38 dist-test-slave-x4qh krb5kdc[22124](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.96.67@KRBTEST.COM for oryx/127.9.96.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 04:58:38.312282 23591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:58:38.304353 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:44517 (local address 127.9.96.67:37333)
0806 04:58:38.304859 (+ 506us) negotiation.cc:107] Waiting for socket to connect
0806 04:58:38.304876 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 04:58:38.304988 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:58:38.305286 (+ 298us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:58:38.305289 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:58:38.305669 (+ 380us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:58:38.305836 (+ 167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:58:38.305843 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:58:38.306867 (+ 1024us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:58:38.306871 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:58:38.307849 (+ 978us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:58:38.307860 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:58:38.308539 (+ 679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:58:38.308543 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:58:38.308667 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:58:38.309296 (+ 629us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:58:38.309315 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:58:38.312051 (+ 2736us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.96.126:44517: Server oryx/127.9.96.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":397,"thread_start_us":263,"threads_started":1}
W20260806 04:58:38.312472 23150 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 04:58:38.304445 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.96.67:37333 (local address 127.9.96.126:44517)
0806 04:58:38.304680 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0806 04:58:38.304686 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 04:58:38.304956 (+ 270us) server_negotiation.cc:407] Connection header received
0806 04:58:38.305085 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:58:38.305088 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:58:38.305149 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:58:38.305229 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:58:38.305955 (+ 726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:38.306731 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:38.307992 (+ 1261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:38.308409 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:38.308509 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:58:38.312246 (+ 3737us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.96.67:37333: BlockingRecv error: recv got EOF from 127.9.96.67:37333 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
Aug 06 04:58:38 dist-test-slave-x4qh krb5kdc[22124](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.96.65@KRBTEST.COM for oryx/127.9.96.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 04:58:38 dist-test-slave-x4qh krb5kdc[22124](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.96.65@KRBTEST.COM for oryx/127.9.96.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 04:58:38.371739 23592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:58:38.363569 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:44517 (local address 127.9.96.65:36387)
0806 04:58:38.364115 (+ 546us) negotiation.cc:107] Waiting for socket to connect
0806 04:58:38.364132 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 04:58:38.364245 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:58:38.364566 (+ 321us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:58:38.364570 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:58:38.364991 (+ 421us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:58:38.365156 (+ 165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:58:38.365165 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:58:38.366180 (+ 1015us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:58:38.366183 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:58:38.367098 (+ 915us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:58:38.367109 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:58:38.367882 (+ 773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:58:38.367887 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:58:38.368016 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:58:38.368668 (+ 652us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:58:38.368686 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:58:38.371482 (+ 2796us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.96.126:44517: Server oryx/127.9.96.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":429,"thread_start_us":297,"threads_started":1}
W20260806 04:58:38.371894 23150 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 04:58:38.363675 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.96.65:36387 (local address 127.9.96.126:44517)
0806 04:58:38.363843 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0806 04:58:38.363847 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 04:58:38.364201 (+ 354us) server_negotiation.cc:407] Connection header received
0806 04:58:38.364344 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:58:38.364347 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:58:38.364431 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:58:38.364512 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:58:38.365279 (+ 767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:38.366043 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:38.367221 (+ 1178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:38.367740 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:38.367833 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:58:38.371634 (+ 3801us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.96.65:36387: BlockingRecv error: recv got EOF from 127.9.96.65:36387 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
Aug 06 04:58:38 dist-test-slave-x4qh krb5kdc[22124](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.96.66@KRBTEST.COM for oryx/127.9.96.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 04:58:38 dist-test-slave-x4qh krb5kdc[22124](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.96.66@KRBTEST.COM for oryx/127.9.96.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 04:58:38.836007 23593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:58:38.827273 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:44517 (local address 127.9.96.66:44563)
0806 04:58:38.827912 (+ 639us) negotiation.cc:107] Waiting for socket to connect
0806 04:58:38.827929 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 04:58:38.828040 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:58:38.828342 (+ 302us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:58:38.828345 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:58:38.828743 (+ 398us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:58:38.828905 (+ 162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:58:38.828913 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:58:38.829864 (+ 951us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:58:38.829868 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:58:38.830766 (+ 898us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:58:38.830777 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:58:38.831572 (+ 795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:58:38.831578 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:58:38.831787 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:58:38.832541 (+ 754us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:58:38.832564 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:58:38.835649 (+ 3085us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.96.126:44517: Server oryx/127.9.96.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":528,"thread_start_us":388,"threads_started":1}
W20260806 04:58:38.836089 23150 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 04:58:38.827489 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.96.66:44563 (local address 127.9.96.126:44517)
0806 04:58:38.827667 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0806 04:58:38.827671 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 04:58:38.828001 (+ 330us) server_negotiation.cc:407] Connection header received
0806 04:58:38.828135 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:58:38.828138 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:58:38.828212 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:58:38.828295 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:58:38.829026 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:38.829729 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:38.830898 (+ 1169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:38.831406 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:38.831536 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:58:38.835846 (+ 4310us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.96.66:44563: BlockingRecv error: recv got EOF from 127.9.96.66:44563 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
Aug 06 04:58:39 dist-test-slave-x4qh krb5kdc[22124](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.96.67@KRBTEST.COM for oryx/127.9.96.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 04:58:39 dist-test-slave-x4qh krb5kdc[22124](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.96.67@KRBTEST.COM for oryx/127.9.96.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 04:58:39.321792 23595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:58:39.313304 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:44517 (local address 127.9.96.67:46463)
0806 04:58:39.313824 (+ 520us) negotiation.cc:107] Waiting for socket to connect
0806 04:58:39.313840 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 04:58:39.314000 (+ 160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:58:39.314346 (+ 346us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:58:39.314353 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:58:39.314754 (+ 401us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:58:39.314910 (+ 156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:58:39.314919 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:58:39.316029 (+ 1110us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:58:39.316035 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:58:39.317135 (+ 1100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:58:39.317150 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:58:39.317984 (+ 834us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:58:39.317991 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:58:39.318157 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:58:39.318792 (+ 635us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:58:39.318811 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:58:39.321560 (+ 2749us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.96.126:44517: Server oryx/127.9.96.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":431,"thread_start_us":268,"threads_started":1}
W20260806 04:58:39.321969 23150 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 04:58:39.313421 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.96.67:46463 (local address 127.9.96.126:44517)
0806 04:58:39.313643 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0806 04:58:39.313649 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 04:58:39.313930 (+ 281us) server_negotiation.cc:407] Connection header received
0806 04:58:39.314124 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:58:39.314131 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:58:39.314196 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:58:39.314300 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:58:39.315060 (+ 760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:39.315888 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:39.317381 (+ 1493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:39.317794 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:39.317938 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:58:39.321743 (+ 3805us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.96.67:46463: BlockingRecv error: recv got EOF from 127.9.96.67:46463 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
Aug 06 04:58:39 dist-test-slave-x4qh krb5kdc[22124](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.96.65@KRBTEST.COM for oryx/127.9.96.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 04:58:39 dist-test-slave-x4qh krb5kdc[22124](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.96.65@KRBTEST.COM for oryx/127.9.96.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 04:58:39.381321 23597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:58:39.372696 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:44517 (local address 127.9.96.65:54091)
0806 04:58:39.373238 (+ 542us) negotiation.cc:107] Waiting for socket to connect
0806 04:58:39.373254 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 04:58:39.373363 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:58:39.373651 (+ 288us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:58:39.373655 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:58:39.373990 (+ 335us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:58:39.374158 (+ 168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:58:39.374165 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:58:39.375231 (+ 1066us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:58:39.375236 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:58:39.376554 (+ 1318us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:58:39.376568 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:58:39.377287 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:58:39.377293 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:58:39.377513 (+ 220us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:58:39.378184 (+ 671us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:58:39.378201 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:58:39.380983 (+ 2782us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.96.126:44517: Server oryx/127.9.96.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":418,"thread_start_us":256,"threads_started":1}
W20260806 04:58:39.381413 23150 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 04:58:39.372756 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.96.65:54091 (local address 127.9.96.126:44517)
0806 04:58:39.373025 (+ 269us) server_negotiation.cc:206] Beginning negotiation
0806 04:58:39.373029 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 04:58:39.373322 (+ 293us) server_negotiation.cc:407] Connection header received
0806 04:58:39.373452 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:58:39.373456 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:58:39.373524 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:58:39.373609 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:58:39.374283 (+ 674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:39.375092 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:39.376697 (+ 1605us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:39.377136 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:39.377272 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:58:39.381174 (+ 3902us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.96.65:54091: BlockingRecv error: recv got EOF from 127.9.96.65:54091 (error 108)
Metrics: {"server-negotiator.queue_time_us":149}
Aug 06 04:58:39 dist-test-slave-x4qh krb5kdc[22124](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.96.66@KRBTEST.COM for oryx/127.9.96.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 04:58:39 dist-test-slave-x4qh krb5kdc[22124](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.96.66@KRBTEST.COM for oryx/127.9.96.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 04:58:39.845130 23599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:58:39.837038 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:44517 (local address 127.9.96.66:53543)
0806 04:58:39.837558 (+ 520us) negotiation.cc:107] Waiting for socket to connect
0806 04:58:39.837574 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 04:58:39.837681 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:58:39.837975 (+ 294us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:58:39.837978 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:58:39.838319 (+ 341us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:58:39.838512 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:58:39.838520 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:58:39.839488 (+ 968us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:58:39.839492 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:58:39.840400 (+ 908us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:58:39.840412 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:58:39.841063 (+ 651us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:58:39.841066 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:58:39.841186 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:58:39.841752 (+ 566us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:58:39.841768 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:58:39.844908 (+ 3140us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.96.126:44517: Server oryx/127.9.96.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":407,"thread_start_us":267,"threads_started":1}
W20260806 04:58:39.845345 23150 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 04:58:39.837146 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.96.66:53543 (local address 127.9.96.126:44517)
0806 04:58:39.837309 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0806 04:58:39.837313 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 04:58:39.837645 (+ 332us) server_negotiation.cc:407] Connection header received
0806 04:58:39.837776 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:58:39.837780 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:58:39.837837 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:58:39.837928 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:58:39.838634 (+ 706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:39.839353 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:39.840520 (+ 1167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:39.840945 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:39.841035 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:58:39.845104 (+ 4069us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.96.66:53543: BlockingRecv error: recv got EOF from 127.9.96.66:53543 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
Aug 06 04:58:40 dist-test-slave-x4qh krb5kdc[22124](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.96.67@KRBTEST.COM for oryx/127.9.96.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 04:58:40 dist-test-slave-x4qh krb5kdc[22124](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.96.67@KRBTEST.COM for oryx/127.9.96.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 04:58:40.331133 23601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:58:40.322720 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:44517 (local address 127.9.96.67:33017)
0806 04:58:40.323232 (+ 512us) negotiation.cc:107] Waiting for socket to connect
0806 04:58:40.323248 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 04:58:40.323358 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:58:40.323760 (+ 402us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:58:40.323763 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:58:40.324129 (+ 366us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:58:40.324297 (+ 168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:58:40.324307 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:58:40.325420 (+ 1113us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:58:40.325423 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:58:40.326372 (+ 949us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:58:40.326382 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:58:40.327359 (+ 977us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:58:40.327364 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:58:40.327543 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:58:40.328116 (+ 573us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:58:40.328132 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:58:40.330852 (+ 2720us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.96.126:44517: Server oryx/127.9.96.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":390,"thread_start_us":281,"threads_started":1}
W20260806 04:58:40.331307 23150 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 04:58:40.322815 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.96.67:33017 (local address 127.9.96.126:44517)
0806 04:58:40.323013 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0806 04:58:40.323017 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 04:58:40.323315 (+ 298us) server_negotiation.cc:407] Connection header received
0806 04:58:40.323537 (+ 222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:58:40.323541 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:58:40.323606 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:58:40.323720 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:58:40.324463 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:40.325237 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:40.326517 (+ 1280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:40.327185 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:40.327336 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:58:40.331074 (+ 3738us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.96.67:33017: BlockingRecv error: recv got EOF from 127.9.96.67:33017 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
Aug 06 04:58:40 dist-test-slave-x4qh krb5kdc[22124](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.96.65@KRBTEST.COM for oryx/127.9.96.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 04:58:40 dist-test-slave-x4qh krb5kdc[22124](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.96.65@KRBTEST.COM for oryx/127.9.96.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 04:58:40.391718 23602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:58:40.382558 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:44517 (local address 127.9.96.65:56307)
0806 04:58:40.383117 (+ 559us) negotiation.cc:107] Waiting for socket to connect
0806 04:58:40.383133 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 04:58:40.383240 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:58:40.383617 (+ 377us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:58:40.383620 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:58:40.384011 (+ 391us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:58:40.384202 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:58:40.384210 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:58:40.385750 (+ 1540us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:58:40.385757 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:58:40.386655 (+ 898us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:58:40.386667 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:58:40.387342 (+ 675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:58:40.387346 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:58:40.387543 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:58:40.388161 (+ 618us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:58:40.388207 (+ 46us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:58:40.391423 (+ 3216us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.96.126:44517: Server oryx/127.9.96.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":434,"thread_start_us":319,"threads_started":1}
W20260806 04:58:40.391793 23150 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 04:58:40.382669 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.96.65:56307 (local address 127.9.96.126:44517)
0806 04:58:40.382835 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0806 04:58:40.382839 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 04:58:40.383209 (+ 370us) server_negotiation.cc:407] Connection header received
0806 04:58:40.383334 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:58:40.383337 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:58:40.383477 (+ 140us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:58:40.383561 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:58:40.384396 (+ 835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:40.385560 (+ 1164us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:40.386779 (+ 1219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:40.387211 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:40.387303 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:58:40.391611 (+ 4308us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.96.65:56307: BlockingRecv error: recv got EOF from 127.9.96.65:56307 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
Aug 06 04:58:40 dist-test-slave-x4qh krb5kdc[22124](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.96.66@KRBTEST.COM for oryx/127.9.96.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 04:58:40 dist-test-slave-x4qh krb5kdc[22124](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.96.66@KRBTEST.COM for oryx/127.9.96.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 04:58:40.854952 23606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:58:40.846216 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:44517 (local address 127.9.96.66:46729)
0806 04:58:40.846822 (+ 606us) negotiation.cc:107] Waiting for socket to connect
0806 04:58:40.846839 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 04:58:40.846998 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:58:40.847358 (+ 360us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:58:40.847362 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:58:40.847818 (+ 456us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:58:40.848019 (+ 201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:58:40.848028 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:58:40.849046 (+ 1018us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:58:40.849057 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:58:40.850015 (+ 958us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:58:40.850027 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:58:40.850818 (+ 791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:58:40.850825 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:58:40.851091 (+ 266us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:58:40.851732 (+ 641us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:58:40.851749 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:58:40.854643 (+ 2894us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.96.126:44517: Server oryx/127.9.96.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":497,"thread_start_us":373,"threads_started":1}
W20260806 04:58:40.855134 23150 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 04:58:40.846318 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.96.66:46729 (local address 127.9.96.126:44517)
0806 04:58:40.846646 (+ 328us) server_negotiation.cc:206] Beginning negotiation
0806 04:58:40.846653 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 04:58:40.846921 (+ 268us) server_negotiation.cc:407] Connection header received
0806 04:58:40.847134 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:58:40.847144 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:58:40.847204 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:58:40.847318 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:58:40.848165 (+ 847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:40.848896 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:40.850165 (+ 1269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:40.850643 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:40.850785 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:58:40.854867 (+ 4082us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.96.66:46729: BlockingRecv error: recv got EOF from 127.9.96.66:46729 (error 108)
Metrics: {"server-negotiator.queue_time_us":172}
I20260806 04:58:41.158922 9601 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu pid 22947
I20260806 04:58:41.170801 23609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:58:41.159941 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44486 (local address 127.9.96.65:39867)
0806 04:58:41.160534 (+ 593us) server_negotiation.cc:206] Beginning negotiation
0806 04:58:41.160540 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 04:58:41.160566 (+ 26us) server_negotiation.cc:407] Connection header received
0806 04:58:41.160623 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:58:41.160629 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:58:41.160838 (+ 209us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:58:41.160983 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:58:41.162331 (+ 1348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:41.163358 (+ 1027us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:41.164739 (+ 1381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:41.165270 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:41.165375 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:58:41.167791 (+ 2416us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:58:41.167800 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:58:41.167812 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:58:41.167841 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:58:41.170324 (+ 2483us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.9.96.65@KRBTEST.COM not found in keytab (ticket kvno 2)
0806 04:58:41.170582 (+ 258us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:44486: Request ticket server kudu/127.9.96.65@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":434,"thread_start_us":303,"threads_started":1}
W20260806 04:58:41.170915 23609 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:44486: Request ticket server kudu/127.9.96.65@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260806 04:58:41.171067 9601 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu pid 22947: Not authorized: Client connection negotiation failed: client connection to 127.9.96.65:39867: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.9.96.65@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260806 04:58:41.171259 9601 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu with pid 22947
I20260806 04:58:41.208892 9601 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu pid 23154
I20260806 04:58:41.221306 23611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:58:41.209906 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47492 (local address 127.9.96.66:32795)
0806 04:58:41.210536 (+ 630us) server_negotiation.cc:206] Beginning negotiation
0806 04:58:41.210543 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 04:58:41.210575 (+ 32us) server_negotiation.cc:407] Connection header received
0806 04:58:41.210637 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:58:41.210644 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:58:41.210926 (+ 282us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:58:41.211087 (+ 161us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:58:41.212396 (+ 1309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:41.213522 (+ 1126us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:41.214759 (+ 1237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:41.215310 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:41.215509 (+ 199us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:58:41.217925 (+ 2416us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:58:41.217933 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:58:41.217946 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:58:41.217979 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:58:41.220793 (+ 2814us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.9.96.66@KRBTEST.COM not found in keytab (ticket kvno 2)
0806 04:58:41.221037 (+ 244us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:47492: Request ticket server kudu/127.9.96.66@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":465,"thread_start_us":313,"threads_started":1}
W20260806 04:58:41.221433 23611 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:47492: Request ticket server kudu/127.9.96.66@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260806 04:58:41.221566 9601 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu pid 23154: Not authorized: Client connection negotiation failed: client connection to 127.9.96.66:32795: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.9.96.66@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260806 04:58:41.221752 9601 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu with pid 23154
I20260806 04:58:41.245131 9601 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu pid 23346
I20260806 04:58:41.256986 23612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:58:41.246042 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57640 (local address 127.9.96.67:34663)
0806 04:58:41.246626 (+ 584us) server_negotiation.cc:206] Beginning negotiation
0806 04:58:41.246633 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 04:58:41.246659 (+ 26us) server_negotiation.cc:407] Connection header received
0806 04:58:41.246720 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:58:41.246727 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:58:41.246953 (+ 226us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:58:41.247123 (+ 170us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:58:41.248433 (+ 1310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:41.249514 (+ 1081us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:41.250730 (+ 1216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:41.251297 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:41.251615 (+ 318us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:58:41.253988 (+ 2373us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:58:41.253996 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:58:41.254009 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:58:41.254040 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:58:41.256458 (+ 2418us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.9.96.67@KRBTEST.COM not found in keytab (ticket kvno 2)
0806 04:58:41.256697 (+ 239us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:57640: Request ticket server kudu/127.9.96.67@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":426,"thread_start_us":316,"threads_started":1}
W20260806 04:58:41.257122 9601 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu pid 23346: Not authorized: Client connection negotiation failed: client connection to 127.9.96.67:34663: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.9.96.67@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260806 04:58:41.257226 23612 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:57640: Request ticket server kudu/127.9.96.67@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260806 04:58:41.257292 9601 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu with pid 23346
I20260806 04:58:41.280125 9601 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu pid 22851
I20260806 04:58:41.281293 22937 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 04:58:41.397900 9601 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu with pid 22851
2026-08-06T04:58:41Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (21441 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-08-06T04:58:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T04:58:41Z Disabled control of system clock
I20260806 04:58:41.487354 9601 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
/tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785992227050689-9601-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785992227050689-9601-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785992227050689-9601-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785992227050689-9601-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.96.126:36131
--webserver_interface=127.9.96.126
--webserver_port=0
--builtin_ntp_servers=127.9.96.84:44083
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.96.126:36131
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260806 04:58:41.815912 23619 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 04:58:41.816246 23619 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 04:58:41.816392 23619 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 04:58:41.823570 23619 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 04:58:41.823731 23619 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 04:58:41.823814 23619 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 04:58:41.823940 23619 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 04:58:41.824021 23619 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 04:58:41.833050 23619 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.96.84:44083
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785992227050689-9601-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785992227050689-9601-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.96.126:36131
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.96.126:36131
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785992227050689-9601-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.96.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785992227050689-9601-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 03:56:12 UTC on ccycloud.quasar-guijbm.root.comops.site
build id 4569618
ASAN enabled
I20260806 04:58:41.834406 23619 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 04:58:41.836328 23619 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 04:58:41.848708 23628 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:58:41.848770 23627 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:58:41.848708 23632 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:58:41.849623 23619 server_base.cc:1034] running on GCE node
I20260806 04:58:41.850404 23619 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 04:58:41.851780 23619 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 04:58:41.853024 23619 hybrid_clock.cc:648] HybridClock initialized: now 1785992321852961 us; error 40 us; skew 500 ppm
I20260806 04:58:41.853420 23619 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 04:58:41.855578 23619 webserver.cc:460] Webserver started at http://127.9.96.126:38499/ using document root <none> and password file <none>
I20260806 04:58:41.856317 23619 fs_manager.cc:359] Metadata directory not provided
I20260806 04:58:41.856436 23619 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 04:58:41.856776 23619 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 04:58:41.859750 23619 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785992227050689-9601-0/minicluster-data/master-0/data/instance:
uuid: "8b7f8ada31e14c3e9857e6fb8ac5fec0"
format_stamp: "Formatted at 2026-08-06 04:58:41 on dist-test-slave-x4qh"
server_key: "039e1978b7d4b02f01a27c024a517642"
server_key_iv: "2b7fe88c1140c769fbe894af4d36e53f"
server_key_version: "encryptionkey@0"
I20260806 04:58:41.860572 23619 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785992227050689-9601-0/minicluster-data/master-0/wal/instance:
uuid: "8b7f8ada31e14c3e9857e6fb8ac5fec0"
format_stamp: "Formatted at 2026-08-06 04:58:41 on dist-test-slave-x4qh"
server_key: "039e1978b7d4b02f01a27c024a517642"
server_key_iv: "2b7fe88c1140c769fbe894af4d36e53f"
server_key_version: "encryptionkey@0"
I20260806 04:58:41.865638 23619 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260806 04:58:41.869065 23641 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 04:58:41.870275 23619 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260806 04:58:41.870457 23619 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785992227050689-9601-0/minicluster-data/master-0/data,/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785992227050689-9601-0/minicluster-data/master-0/wal
uuid: "8b7f8ada31e14c3e9857e6fb8ac5fec0"
format_stamp: "Formatted at 2026-08-06 04:58:41 on dist-test-slave-x4qh"
server_key: "039e1978b7d4b02f01a27c024a517642"
server_key_iv: "2b7fe88c1140c769fbe894af4d36e53f"
server_key_version: "encryptionkey@0"
I20260806 04:58:41.870678 23619 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785992227050689-9601-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785992227050689-9601-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785992227050689-9601-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 04:58:41.892724 23619 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 04:58:41.893527 23619 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 04:58:41.893821 23619 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 04:58:41.912341 23619 rpc_server.cc:307] RPC server started. Bound to: 127.9.96.126:36131
I20260806 04:58:41.912366 23708 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.96.126:36131 every 8 connection(s)
I20260806 04:58:41.913695 23619 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785992227050689-9601-0/minicluster-data/master-0/data/info.pb
I20260806 04:58:41.917183 9601 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu as pid 23619
I20260806 04:58:41.917338 23709 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 04:58:41.917429 9601 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785992227050689-9601-0/minicluster-data/master-0/wal/instance
I20260806 04:58:41.918112 9601 external_mini_cluster.cc:1428] Setting key 29b433529dfe9a052b885628607b5c68
I20260806 04:58:41.931857 23709 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8b7f8ada31e14c3e9857e6fb8ac5fec0: Bootstrap starting.
I20260806 04:58:41.934060 23714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:58:41.920232 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59380 (local address 127.9.96.126:36131)
0806 04:58:41.921321 (+ 1089us) server_negotiation.cc:206] Beginning negotiation
0806 04:58:41.921335 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0806 04:58:41.921390 (+ 55us) server_negotiation.cc:407] Connection header received
0806 04:58:41.924117 (+ 2727us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:58:41.924146 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:58:41.924733 (+ 587us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:58:41.925185 (+ 452us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:58:41.925695 (+ 510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:41.927603 (+ 1908us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:41.928914 (+ 1311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:41.929999 (+ 1085us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:41.930158 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:58:41.931289 (+ 1131us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:58:41.931299 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:58:41.931316 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 04:58:41.932032 (+ 716us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:58:41.932139 (+ 107us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 04:58:41.932167 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:58:41.932283 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0806 04:58:41.932472 (+ 189us) server_negotiation.cc:299] Negotiation successful
0806 04:58:41.932782 (+ 310us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":572,"thread_start_us":344,"threads_started":1}
I20260806 04:58:41.935542 23709 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8b7f8ada31e14c3e9857e6fb8ac5fec0: Neither blocks nor log segments found. Creating new log.
I20260806 04:58:41.936612 23709 log.cc:824] T 00000000000000000000000000000000 P 8b7f8ada31e14c3e9857e6fb8ac5fec0: Log is configured to *not* fsync() on all Append() calls
I20260806 04:58:41.939558 23709 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8b7f8ada31e14c3e9857e6fb8ac5fec0: No bootstrap required, opened a new log
I20260806 04:58:41.946743 23709 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8b7f8ada31e14c3e9857e6fb8ac5fec0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8b7f8ada31e14c3e9857e6fb8ac5fec0" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 36131 } }
I20260806 04:58:41.947041 23709 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8b7f8ada31e14c3e9857e6fb8ac5fec0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 04:58:41.947106 23709 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8b7f8ada31e14c3e9857e6fb8ac5fec0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8b7f8ada31e14c3e9857e6fb8ac5fec0, State: Initialized, Role: FOLLOWER
I20260806 04:58:41.947690 23709 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8b7f8ada31e14c3e9857e6fb8ac5fec0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8b7f8ada31e14c3e9857e6fb8ac5fec0" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 36131 } }
I20260806 04:58:41.947885 23709 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8b7f8ada31e14c3e9857e6fb8ac5fec0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 04:58:41.947979 23709 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8b7f8ada31e14c3e9857e6fb8ac5fec0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 04:58:41.948102 23709 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8b7f8ada31e14c3e9857e6fb8ac5fec0 [term 0 FOLLOWER]: Advancing to term 1
I20260806 04:58:41.950171 23709 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8b7f8ada31e14c3e9857e6fb8ac5fec0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8b7f8ada31e14c3e9857e6fb8ac5fec0" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 36131 } }
I20260806 04:58:41.950594 23709 leader_election.cc:302] T 00000000000000000000000000000000 P 8b7f8ada31e14c3e9857e6fb8ac5fec0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8b7f8ada31e14c3e9857e6fb8ac5fec0; no voters:
I20260806 04:58:41.951102 23709 leader_election.cc:288] T 00000000000000000000000000000000 P 8b7f8ada31e14c3e9857e6fb8ac5fec0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 04:58:41.951261 23716 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8b7f8ada31e14c3e9857e6fb8ac5fec0 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 04:58:41.952054 23716 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8b7f8ada31e14c3e9857e6fb8ac5fec0 [term 1 LEADER]: Becoming Leader. State: Replica: 8b7f8ada31e14c3e9857e6fb8ac5fec0, State: Running, Role: LEADER
I20260806 04:58:41.952633 23716 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8b7f8ada31e14c3e9857e6fb8ac5fec0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8b7f8ada31e14c3e9857e6fb8ac5fec0" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 36131 } }
I20260806 04:58:41.953214 23709 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8b7f8ada31e14c3e9857e6fb8ac5fec0 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 04:58:41.955535 23717 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8b7f8ada31e14c3e9857e6fb8ac5fec0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8b7f8ada31e14c3e9857e6fb8ac5fec0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8b7f8ada31e14c3e9857e6fb8ac5fec0" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 36131 } } }
I20260806 04:58:41.956002 23717 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8b7f8ada31e14c3e9857e6fb8ac5fec0 [sys.catalog]: This master's current role is: LEADER
I20260806 04:58:41.956053 23719 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8b7f8ada31e14c3e9857e6fb8ac5fec0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8b7f8ada31e14c3e9857e6fb8ac5fec0. Latest consensus state: current_term: 1 leader_uuid: "8b7f8ada31e14c3e9857e6fb8ac5fec0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8b7f8ada31e14c3e9857e6fb8ac5fec0" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 36131 } } }
I20260806 04:58:41.956432 23719 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8b7f8ada31e14c3e9857e6fb8ac5fec0 [sys.catalog]: This master's current role is: LEADER
I20260806 04:58:41.957921 23723 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 04:58:41.963924 23723 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 04:58:41.972744 23723 catalog_manager.cc:1347] Generated new cluster ID: 4a0b5019a68640839455ca90c554ab8b
I20260806 04:58:41.972856 23723 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 04:58:41.989675 23723 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 04:58:41.991555 23723 catalog_manager.cc:1504] Loading token signing keys...
I20260806 04:58:42.003041 23723 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8b7f8ada31e14c3e9857e6fb8ac5fec0: Generated new TSK 0
I20260806 04:58:42.004081 23723 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 04:58:42.009088 9601 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
/tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785992227050689-9601-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785992227050689-9601-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785992227050689-9601-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785992227050689-9601-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.96.65:0
--local_ip_for_outbound_sockets=127.9.96.65
--webserver_interface=127.9.96.65
--webserver_port=0
--tserver_master_addrs=127.9.96.126:36131
--builtin_ntp_servers=127.9.96.84:44083
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260806 04:58:42.338826 23742 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 04:58:42.339139 23742 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 04:58:42.339303 23742 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 04:58:42.346504 23742 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 04:58:42.346647 23742 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 04:58:42.346818 23742 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.96.65
I20260806 04:58:42.356015 23742 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.96.84:44083
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785992227050689-9601-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785992227050689-9601-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.96.65:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785992227050689-9601-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.96.65
--webserver_port=0
--tserver_master_addrs=127.9.96.126:36131
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.96.65
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785992227050689-9601-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 03:56:12 UTC on ccycloud.quasar-guijbm.root.comops.site
build id 4569618
ASAN enabled
I20260806 04:58:42.357245 23742 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 04:58:42.358996 23742 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 04:58:42.371868 23749 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:58:42.371944 23748 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:58:42.371970 23751 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:58:42.372980 23742 server_base.cc:1034] running on GCE node
I20260806 04:58:42.373582 23742 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 04:58:42.374527 23742 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 04:58:42.375772 23742 hybrid_clock.cc:648] HybridClock initialized: now 1785992322375714 us; error 69 us; skew 500 ppm
I20260806 04:58:42.376089 23742 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 04:58:42.381991 23742 webserver.cc:460] Webserver started at http://127.9.96.65:44603/ using document root <none> and password file <none>
I20260806 04:58:42.382699 23742 fs_manager.cc:359] Metadata directory not provided
I20260806 04:58:42.382838 23742 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 04:58:42.383184 23742 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 04:58:42.386078 23742 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785992227050689-9601-0/minicluster-data/ts-0/data/instance:
uuid: "0eda9a1b945747dbb12960d1554275c8"
format_stamp: "Formatted at 2026-08-06 04:58:42 on dist-test-slave-x4qh"
server_key: "6a35e715f18776800a70d930a61d0586"
server_key_iv: "5118753c12dc20ccc1c67a4150322177"
server_key_version: "encryptionkey@0"
I20260806 04:58:42.386897 23742 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785992227050689-9601-0/minicluster-data/ts-0/wal/instance:
uuid: "0eda9a1b945747dbb12960d1554275c8"
format_stamp: "Formatted at 2026-08-06 04:58:42 on dist-test-slave-x4qh"
server_key: "6a35e715f18776800a70d930a61d0586"
server_key_iv: "5118753c12dc20ccc1c67a4150322177"
server_key_version: "encryptionkey@0"
I20260806 04:58:42.392058 23742 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260806 04:58:42.395849 23760 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 04:58:42.397177 23742 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.003s
I20260806 04:58:42.397306 23742 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785992227050689-9601-0/minicluster-data/ts-0/data,/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785992227050689-9601-0/minicluster-data/ts-0/wal
uuid: "0eda9a1b945747dbb12960d1554275c8"
format_stamp: "Formatted at 2026-08-06 04:58:42 on dist-test-slave-x4qh"
server_key: "6a35e715f18776800a70d930a61d0586"
server_key_iv: "5118753c12dc20ccc1c67a4150322177"
server_key_version: "encryptionkey@0"
I20260806 04:58:42.397467 23742 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785992227050689-9601-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785992227050689-9601-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785992227050689-9601-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 04:58:42.412184 23742 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 04:58:42.412897 23742 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 04:58:42.413123 23742 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 04:58:42.413946 23742 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 04:58:42.415298 23742 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 04:58:42.415429 23742 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 04:58:42.415534 23742 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 04:58:42.415594 23742 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 04:58:42.448220 23742 rpc_server.cc:307] RPC server started. Bound to: 127.9.96.65:40573
I20260806 04:58:42.448321 23923 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.96.65:40573 every 8 connection(s)
I20260806 04:58:42.449468 23742 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785992227050689-9601-0/minicluster-data/ts-0/data/info.pb
I20260806 04:58:42.456525 9601 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu as pid 23742
I20260806 04:58:42.456770 9601 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785992227050689-9601-0/minicluster-data/ts-0/wal/instance
I20260806 04:58:42.457479 9601 external_mini_cluster.cc:1428] Setting key 401fcd3fdbad5caa205af31a8c372fac
I20260806 04:58:42.460216 9601 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
/tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785992227050689-9601-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785992227050689-9601-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785992227050689-9601-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785992227050689-9601-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.96.66:0
--local_ip_for_outbound_sockets=127.9.96.66
--webserver_interface=127.9.96.66
--webserver_port=0
--tserver_master_addrs=127.9.96.126:36131
--builtin_ntp_servers=127.9.96.84:44083
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260806 04:58:42.462221 23926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:58:42.451953 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.96.65:36119 (local address 127.9.96.126:36131)
0806 04:58:42.452407 (+ 454us) server_negotiation.cc:206] Beginning negotiation
0806 04:58:42.452411 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 04:58:42.453647 (+ 1236us) server_negotiation.cc:407] Connection header received
0806 04:58:42.455545 (+ 1898us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:58:42.455550 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:58:42.455640 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:58:42.455760 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:58:42.456812 (+ 1052us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:42.457787 (+ 975us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:42.459323 (+ 1536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:42.459834 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:42.459972 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:58:42.461373 (+ 1401us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:58:42.461377 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:58:42.461381 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 04:58:42.461397 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:58:42.461433 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 04:58:42.461445 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:58:42.461538 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0806 04:58:42.461796 (+ 258us) server_negotiation.cc:299] Negotiation successful
0806 04:58:42.461889 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":340,"thread_start_us":235,"threads_started":1}
I20260806 04:58:42.463135 23927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:58:42.452507 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:36131 (local address 127.9.96.65:36119)
0806 04:58:42.453483 (+ 976us) negotiation.cc:107] Waiting for socket to connect
0806 04:58:42.453525 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0806 04:58:42.455239 (+ 1714us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:58:42.455994 (+ 755us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:58:42.456001 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:58:42.456077 (+ 76us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:58:42.456614 (+ 537us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:58:42.456629 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:58:42.457955 (+ 1326us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:58:42.457960 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:58:42.459174 (+ 1214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:58:42.459186 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:58:42.460076 (+ 890us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:58:42.460085 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:58:42.460309 (+ 224us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:58:42.461069 (+ 760us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 04:58:42.461071 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:58:42.461106 (+ 35us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 04:58:42.461171 (+ 65us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 04:58:42.461553 (+ 382us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 04:58:42.461566 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 04:58:42.461573 (+ 7us) client_negotiation.cc:769] Sending connection context
0806 04:58:42.461755 (+ 182us) client_negotiation.cc:240] Negotiation successful
0806 04:58:42.462110 (+ 355us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":468,"thread_start_us":306,"threads_started":1}
I20260806 04:58:42.464728 23924 heartbeater.cc:347] Connected to a master server at 127.9.96.126:36131
I20260806 04:58:42.465018 23924 heartbeater.cc:464] Registering TS with master...
I20260806 04:58:42.465898 23924 heartbeater.cc:511] Master 127.9.96.126:36131 requested a full tablet report, sending...
I20260806 04:58:42.468114 23664 ts_manager.cc:194] Registered new tserver with Master: 0eda9a1b945747dbb12960d1554275c8 (127.9.96.65:40573)
I20260806 04:58:42.470245 23664 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.9.96.65:36119
W20260806 04:58:42.788103 23929 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 04:58:42.788348 23929 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 04:58:42.788517 23929 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 04:58:42.795460 23929 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 04:58:42.795575 23929 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 04:58:42.795745 23929 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.96.66
I20260806 04:58:42.804680 23929 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.96.84:44083
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785992227050689-9601-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785992227050689-9601-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.96.66:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785992227050689-9601-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.96.66
--webserver_port=0
--tserver_master_addrs=127.9.96.126:36131
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.96.66
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785992227050689-9601-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 03:56:12 UTC on ccycloud.quasar-guijbm.root.comops.site
build id 4569618
ASAN enabled
I20260806 04:58:42.805886 23929 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 04:58:42.807714 23929 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 04:58:42.820614 23935 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:58:42.820755 23934 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:58:42.820696 23937 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:58:42.821498 23929 server_base.cc:1034] running on GCE node
I20260806 04:58:42.822088 23929 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 04:58:42.823002 23929 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 04:58:42.824227 23929 hybrid_clock.cc:648] HybridClock initialized: now 1785992322824182 us; error 33 us; skew 500 ppm
I20260806 04:58:42.824558 23929 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 04:58:42.826660 23929 webserver.cc:460] Webserver started at http://127.9.96.66:41151/ using document root <none> and password file <none>
I20260806 04:58:42.827316 23929 fs_manager.cc:359] Metadata directory not provided
I20260806 04:58:42.827487 23929 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 04:58:42.827819 23929 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 04:58:42.830556 23929 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785992227050689-9601-0/minicluster-data/ts-1/data/instance:
uuid: "a10bc069f1f64308aed072fed6fb6c09"
format_stamp: "Formatted at 2026-08-06 04:58:42 on dist-test-slave-x4qh"
server_key: "5b6d9d0bc310350d6389541375c588fa"
server_key_iv: "8e97204cf777ee13fe8f7be93de6ad14"
server_key_version: "encryptionkey@0"
I20260806 04:58:42.831424 23929 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785992227050689-9601-0/minicluster-data/ts-1/wal/instance:
uuid: "a10bc069f1f64308aed072fed6fb6c09"
format_stamp: "Formatted at 2026-08-06 04:58:42 on dist-test-slave-x4qh"
server_key: "5b6d9d0bc310350d6389541375c588fa"
server_key_iv: "8e97204cf777ee13fe8f7be93de6ad14"
server_key_version: "encryptionkey@0"
I20260806 04:58:42.836265 23929 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260806 04:58:42.839470 23945 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 04:58:42.840701 23929 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260806 04:58:42.840880 23929 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785992227050689-9601-0/minicluster-data/ts-1/data,/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785992227050689-9601-0/minicluster-data/ts-1/wal
uuid: "a10bc069f1f64308aed072fed6fb6c09"
format_stamp: "Formatted at 2026-08-06 04:58:42 on dist-test-slave-x4qh"
server_key: "5b6d9d0bc310350d6389541375c588fa"
server_key_iv: "8e97204cf777ee13fe8f7be93de6ad14"
server_key_version: "encryptionkey@0"
I20260806 04:58:42.841090 23929 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785992227050689-9601-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785992227050689-9601-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785992227050689-9601-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 04:58:42.858299 23929 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 04:58:42.859048 23929 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 04:58:42.859323 23929 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 04:58:42.860275 23929 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 04:58:42.861692 23929 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 04:58:42.861778 23929 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 04:58:42.861858 23929 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 04:58:42.861917 23929 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 04:58:42.895169 23929 rpc_server.cc:307] RPC server started. Bound to: 127.9.96.66:35523
I20260806 04:58:42.895187 24109 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.96.66:35523 every 8 connection(s)
I20260806 04:58:42.896523 23929 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785992227050689-9601-0/minicluster-data/ts-1/data/info.pb
I20260806 04:58:42.897555 9601 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu as pid 23929
I20260806 04:58:42.897771 9601 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785992227050689-9601-0/minicluster-data/ts-1/wal/instance
I20260806 04:58:42.898300 9601 external_mini_cluster.cc:1428] Setting key 7147b721e93a1f2749a37e395fefa2d0
I20260806 04:58:42.901283 9601 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
/tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785992227050689-9601-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785992227050689-9601-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785992227050689-9601-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785992227050689-9601-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.96.67:0
--local_ip_for_outbound_sockets=127.9.96.67
--webserver_interface=127.9.96.67
--webserver_port=0
--tserver_master_addrs=127.9.96.126:36131
--builtin_ntp_servers=127.9.96.84:44083
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260806 04:58:42.910076 23926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:58:42.899359 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.96.66:53355 (local address 127.9.96.126:36131)
0806 04:58:42.899597 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0806 04:58:42.899603 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 04:58:42.901304 (+ 1701us) server_negotiation.cc:407] Connection header received
0806 04:58:42.903114 (+ 1810us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:58:42.903122 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:58:42.903212 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:58:42.903321 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:58:42.904457 (+ 1136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:42.905561 (+ 1104us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:42.907044 (+ 1483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:42.907755 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:42.907910 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:58:42.909166 (+ 1256us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:58:42.909175 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:58:42.909182 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 04:58:42.909206 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:58:42.909246 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 04:58:42.909265 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:58:42.909366 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0806 04:58:42.909567 (+ 201us) server_negotiation.cc:299] Negotiation successful
0806 04:58:42.909674 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260806 04:58:42.910888 24112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:58:42.899966 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:36131 (local address 127.9.96.66:53355)
0806 04:58:42.901132 (+ 1166us) negotiation.cc:107] Waiting for socket to connect
0806 04:58:42.901184 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0806 04:58:42.902855 (+ 1671us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:58:42.903578 (+ 723us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:58:42.903586 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:58:42.903677 (+ 91us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:58:42.904260 (+ 583us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:58:42.904275 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:58:42.905723 (+ 1448us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:58:42.905727 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:58:42.906894 (+ 1167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:58:42.906911 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:58:42.907960 (+ 1049us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:58:42.907965 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:58:42.908156 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:58:42.908864 (+ 708us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 04:58:42.908866 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:58:42.908902 (+ 36us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 04:58:42.908972 (+ 70us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 04:58:42.909376 (+ 404us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 04:58:42.909383 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 04:58:42.909389 (+ 6us) client_negotiation.cc:769] Sending connection context
0806 04:58:42.909522 (+ 133us) client_negotiation.cc:240] Negotiation successful
0806 04:58:42.909780 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":674,"thread_start_us":447,"threads_started":1}
I20260806 04:58:42.912457 24110 heartbeater.cc:347] Connected to a master server at 127.9.96.126:36131
I20260806 04:58:42.912806 24110 heartbeater.cc:464] Registering TS with master...
I20260806 04:58:42.913534 24110 heartbeater.cc:511] Master 127.9.96.126:36131 requested a full tablet report, sending...
I20260806 04:58:42.915097 23664 ts_manager.cc:194] Registered new tserver with Master: a10bc069f1f64308aed072fed6fb6c09 (127.9.96.66:35523)
I20260806 04:58:42.916200 23664 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.9.96.66:53355
W20260806 04:58:43.235703 24113 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 04:58:43.236007 24113 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 04:58:43.236171 24113 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 04:58:43.243134 24113 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 04:58:43.243261 24113 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 04:58:43.243453 24113 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.96.67
I20260806 04:58:43.252326 24113 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.96.84:44083
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785992227050689-9601-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785992227050689-9601-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.96.67:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785992227050689-9601-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.96.67
--webserver_port=0
--tserver_master_addrs=127.9.96.126:36131
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.96.67
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785992227050689-9601-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 03:56:12 UTC on ccycloud.quasar-guijbm.root.comops.site
build id 4569618
ASAN enabled
I20260806 04:58:43.253499 24113 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 04:58:43.255210 24113 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 04:58:43.268348 24119 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:58:43.268433 24118 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:58:43.268405 24122 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:58:43.269209 24113 server_base.cc:1034] running on GCE node
I20260806 04:58:43.269826 24113 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 04:58:43.270733 24113 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 04:58:43.271934 24113 hybrid_clock.cc:648] HybridClock initialized: now 1785992323271898 us; error 28 us; skew 500 ppm
I20260806 04:58:43.272253 24113 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 04:58:43.274547 24113 webserver.cc:460] Webserver started at http://127.9.96.67:41251/ using document root <none> and password file <none>
I20260806 04:58:43.275259 24113 fs_manager.cc:359] Metadata directory not provided
I20260806 04:58:43.275380 24113 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 04:58:43.275697 24113 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 04:58:43.278512 24113 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785992227050689-9601-0/minicluster-data/ts-2/data/instance:
uuid: "ed1c0891035f4674998d5dc770e17da4"
format_stamp: "Formatted at 2026-08-06 04:58:43 on dist-test-slave-x4qh"
server_key: "316324e8e8cdcd6af5f1e3f299162e93"
server_key_iv: "5f5216011d1c1e14deb564ce1ee0e7b2"
server_key_version: "encryptionkey@0"
I20260806 04:58:43.279261 24113 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785992227050689-9601-0/minicluster-data/ts-2/wal/instance:
uuid: "ed1c0891035f4674998d5dc770e17da4"
format_stamp: "Formatted at 2026-08-06 04:58:43 on dist-test-slave-x4qh"
server_key: "316324e8e8cdcd6af5f1e3f299162e93"
server_key_iv: "5f5216011d1c1e14deb564ce1ee0e7b2"
server_key_version: "encryptionkey@0"
I20260806 04:58:43.284102 24113 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260806 04:58:43.287621 24131 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 04:58:43.288848 24113 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260806 04:58:43.288996 24113 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785992227050689-9601-0/minicluster-data/ts-2/data,/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785992227050689-9601-0/minicluster-data/ts-2/wal
uuid: "ed1c0891035f4674998d5dc770e17da4"
format_stamp: "Formatted at 2026-08-06 04:58:43 on dist-test-slave-x4qh"
server_key: "316324e8e8cdcd6af5f1e3f299162e93"
server_key_iv: "5f5216011d1c1e14deb564ce1ee0e7b2"
server_key_version: "encryptionkey@0"
I20260806 04:58:43.289170 24113 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785992227050689-9601-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785992227050689-9601-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785992227050689-9601-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 04:58:43.309254 24113 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 04:58:43.310124 24113 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 04:58:43.310364 24113 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 04:58:43.311259 24113 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 04:58:43.312772 24113 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 04:58:43.312868 24113 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 04:58:43.312950 24113 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 04:58:43.313005 24113 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 04:58:43.346426 24113 rpc_server.cc:307] RPC server started. Bound to: 127.9.96.67:33153
I20260806 04:58:43.346515 24295 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.96.67:33153 every 8 connection(s)
I20260806 04:58:43.347851 24113 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785992227050689-9601-0/minicluster-data/ts-2/data/info.pb
I20260806 04:58:43.350888 9601 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu as pid 24113
I20260806 04:58:43.351083 9601 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785992227050689-9601-0/minicluster-data/ts-2/wal/instance
I20260806 04:58:43.351706 9601 external_mini_cluster.cc:1428] Setting key 1b490ec2c2e7e740dfdbc9d8b33c04b9
I20260806 04:58:43.360569 23926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:58:43.350310 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.96.67:50581 (local address 127.9.96.126:36131)
0806 04:58:43.350563 (+ 253us) server_negotiation.cc:206] Beginning negotiation
0806 04:58:43.350570 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 04:58:43.352245 (+ 1675us) server_negotiation.cc:407] Connection header received
0806 04:58:43.354326 (+ 2081us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:58:43.354330 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:58:43.354432 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:58:43.354517 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:58:43.355743 (+ 1226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:43.356565 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:43.358062 (+ 1497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:43.358461 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:43.358551 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:58:43.359904 (+ 1353us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:58:43.359910 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:58:43.359917 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 04:58:43.359939 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:58:43.359968 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 04:58:43.359980 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:58:43.360048 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0806 04:58:43.360261 (+ 213us) server_negotiation.cc:299] Negotiation successful
0806 04:58:43.360338 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
I20260806 04:58:43.361506 24300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:58:43.350862 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:36131 (local address 127.9.96.67:50581)
0806 04:58:43.352041 (+ 1179us) negotiation.cc:107] Waiting for socket to connect
0806 04:58:43.352099 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0806 04:58:43.354061 (+ 1962us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:58:43.354768 (+ 707us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:58:43.354783 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:58:43.354898 (+ 115us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:58:43.355596 (+ 698us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:58:43.355615 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:58:43.356689 (+ 1074us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:58:43.356693 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:58:43.357935 (+ 1242us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:58:43.357947 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:58:43.358635 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:58:43.358639 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:58:43.358839 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:58:43.359591 (+ 752us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 04:58:43.359594 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:58:43.359630 (+ 36us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 04:58:43.359703 (+ 73us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 04:58:43.360068 (+ 365us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 04:58:43.360076 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 04:58:43.360082 (+ 6us) client_negotiation.cc:769] Sending connection context
0806 04:58:43.360217 (+ 135us) client_negotiation.cc:240] Negotiation successful
0806 04:58:43.360477 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":608,"thread_start_us":427,"threads_started":1}
I20260806 04:58:43.362747 24296 heartbeater.cc:347] Connected to a master server at 127.9.96.126:36131
I20260806 04:58:43.363037 24296 heartbeater.cc:464] Registering TS with master...
I20260806 04:58:43.363783 24296 heartbeater.cc:511] Master 127.9.96.126:36131 requested a full tablet report, sending...
I20260806 04:58:43.365094 23664 ts_manager.cc:194] Registered new tserver with Master: ed1c0891035f4674998d5dc770e17da4 (127.9.96.67:33153)
I20260806 04:58:43.366123 23664 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.9.96.67:50581
I20260806 04:58:43.367619 9601 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260806 04:58:43.373793 23926 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 04:58:43.372191 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59390 (local address 127.9.96.126:36131)
0806 04:58:43.372400 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0806 04:58:43.372404 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 04:58:43.372938 (+ 534us) server_negotiation.cc:407] Connection header received
0806 04:58:43.373059 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:58:43.373062 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:58:43.373129 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:58:43.373210 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:58:43.373611 (+ 401us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59390: BlockingRecv error: recv got EOF from 127.0.0.1:59390 (error 108)
Metrics: {"server-negotiator.queue_time_us":95}
I20260806 04:58:43.374451 9601 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu pid 23742
I20260806 04:58:43.382128 24310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:58:43.375470 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44890 (local address 127.9.96.65:40573)
0806 04:58:43.375990 (+ 520us) server_negotiation.cc:206] Beginning negotiation
0806 04:58:43.375999 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 04:58:43.376029 (+ 30us) server_negotiation.cc:407] Connection header received
0806 04:58:43.376087 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:58:43.376092 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:58:43.376249 (+ 157us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:58:43.376418 (+ 169us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:58:43.376821 (+ 403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:43.377805 (+ 984us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:43.379017 (+ 1212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:43.379620 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:43.379758 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:58:43.380828 (+ 1070us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:58:43.380835 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:58:43.380847 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 04:58:43.381353 (+ 506us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:58:43.381401 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 04:58:43.381420 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:58:43.381541 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0806 04:58:43.381677 (+ 136us) server_negotiation.cc:299] Negotiation successful
0806 04:58:43.381817 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":365,"thread_start_us":247,"threads_started":1}
I20260806 04:58:43.383312 23915 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 04:58:43.464459 9601 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu with pid 23742
I20260806 04:58:43.487550 9601 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu pid 23929
I20260806 04:58:43.495220 24312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:58:43.488511 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48998 (local address 127.9.96.66:35523)
0806 04:58:43.489103 (+ 592us) server_negotiation.cc:206] Beginning negotiation
0806 04:58:43.489110 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 04:58:43.489136 (+ 26us) server_negotiation.cc:407] Connection header received
0806 04:58:43.489196 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:58:43.489203 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:58:43.489432 (+ 229us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:58:43.489557 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:58:43.489896 (+ 339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:43.490883 (+ 987us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:43.492209 (+ 1326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:43.492842 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:43.492943 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:58:43.493939 (+ 996us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:58:43.493958 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:58:43.493972 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 04:58:43.494548 (+ 576us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:58:43.494592 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 04:58:43.494611 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:58:43.494696 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0806 04:58:43.494832 (+ 136us) server_negotiation.cc:299] Negotiation successful
0806 04:58:43.494945 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":430,"thread_start_us":313,"threads_started":1}
I20260806 04:58:43.496395 24105 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 04:58:43.579130 9601 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu with pid 23929
I20260806 04:58:43.601696 9601 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu pid 24113
I20260806 04:58:43.610282 24315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:58:43.602645 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34162 (local address 127.9.96.67:33153)
0806 04:58:43.603295 (+ 650us) server_negotiation.cc:206] Beginning negotiation
0806 04:58:43.603304 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 04:58:43.603336 (+ 32us) server_negotiation.cc:407] Connection header received
0806 04:58:43.603484 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:58:43.603491 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:58:43.603748 (+ 257us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:58:43.603934 (+ 186us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:58:43.604446 (+ 512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:43.605501 (+ 1055us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:43.606757 (+ 1256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:43.607342 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:43.607647 (+ 305us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:58:43.608819 (+ 1172us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:58:43.608828 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:58:43.608841 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 04:58:43.609405 (+ 564us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:58:43.609463 (+ 58us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 04:58:43.609481 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:58:43.609607 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0806 04:58:43.609776 (+ 169us) server_negotiation.cc:299] Negotiation successful
0806 04:58:43.609909 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":494,"thread_start_us":339,"threads_started":1}
I20260806 04:58:43.611634 24291 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 04:58:43.693621 9601 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu with pid 24113
I20260806 04:58:43.716943 9601 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu pid 23619
I20260806 04:58:43.718106 23704 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 04:58:43.817557 9601 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu with pid 23619
2026-08-06T04:58:43Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (2411 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-08-06T04:58:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T04:58:43Z Disabled control of system clock
I20260806 04:58:43.901248 9601 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
/tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785992227050689-9601-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785992227050689-9601-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785992227050689-9601-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785992227050689-9601-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.96.126:40141
--webserver_interface=127.9.96.126
--webserver_port=0
--builtin_ntp_servers=127.9.96.84:33819
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.96.126:40141
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260806 04:58:44.226317 24326 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 04:58:44.226583 24326 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 04:58:44.226672 24326 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 04:58:44.233420 24326 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 04:58:44.233537 24326 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 04:58:44.233582 24326 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 04:58:44.233632 24326 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 04:58:44.233683 24326 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 04:58:44.242121 24326 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.96.84:33819
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785992227050689-9601-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785992227050689-9601-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.96.126:40141
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.96.126:40141
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785992227050689-9601-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.96.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785992227050689-9601-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 03:56:12 UTC on ccycloud.quasar-guijbm.root.comops.site
build id 4569618
ASAN enabled
I20260806 04:58:44.243239 24326 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 04:58:44.245041 24326 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 04:58:44.256898 24336 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:58:44.256979 24335 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:58:44.257004 24339 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:58:44.257970 24326 server_base.cc:1034] running on GCE node
I20260806 04:58:44.258623 24326 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 04:58:44.259883 24326 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 04:58:44.261113 24326 hybrid_clock.cc:648] HybridClock initialized: now 1785992324261084 us; error 43 us; skew 500 ppm
I20260806 04:58:44.261456 24326 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 04:58:44.263746 24326 webserver.cc:460] Webserver started at http://127.9.96.126:34793/ using document root <none> and password file <none>
I20260806 04:58:44.264453 24326 fs_manager.cc:359] Metadata directory not provided
I20260806 04:58:44.264592 24326 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 04:58:44.264925 24326 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 04:58:44.267750 24326 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785992227050689-9601-0/minicluster-data/master-0/data/instance:
uuid: "2f0a6eca06674cf0bae68ef844fa0c89"
format_stamp: "Formatted at 2026-08-06 04:58:44 on dist-test-slave-x4qh"
server_key: "88db25e6655c59cffcf40cc4898f1f40"
server_key_iv: "6ccc15f93ef6547a4119fda389c17d27"
server_key_version: "encryptionkey@0"
I20260806 04:58:44.268530 24326 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785992227050689-9601-0/minicluster-data/master-0/wal/instance:
uuid: "2f0a6eca06674cf0bae68ef844fa0c89"
format_stamp: "Formatted at 2026-08-06 04:58:44 on dist-test-slave-x4qh"
server_key: "88db25e6655c59cffcf40cc4898f1f40"
server_key_iv: "6ccc15f93ef6547a4119fda389c17d27"
server_key_version: "encryptionkey@0"
I20260806 04:58:44.273538 24326 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.003s
I20260806 04:58:44.276840 24348 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 04:58:44.277958 24326 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260806 04:58:44.278152 24326 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785992227050689-9601-0/minicluster-data/master-0/data,/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785992227050689-9601-0/minicluster-data/master-0/wal
uuid: "2f0a6eca06674cf0bae68ef844fa0c89"
format_stamp: "Formatted at 2026-08-06 04:58:44 on dist-test-slave-x4qh"
server_key: "88db25e6655c59cffcf40cc4898f1f40"
server_key_iv: "6ccc15f93ef6547a4119fda389c17d27"
server_key_version: "encryptionkey@0"
I20260806 04:58:44.278388 24326 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785992227050689-9601-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785992227050689-9601-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785992227050689-9601-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 04:58:44.284098 24326 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 04:58:44.284839 24326 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 04:58:44.285140 24326 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 04:58:44.303408 24326 rpc_server.cc:307] RPC server started. Bound to: 127.9.96.126:40141
I20260806 04:58:44.303490 24420 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.96.126:40141 every 8 connection(s)
I20260806 04:58:44.304675 24326 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785992227050689-9601-0/minicluster-data/master-0/data/info.pb
I20260806 04:58:44.308210 24421 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 04:58:44.311643 9601 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu as pid 24326
I20260806 04:58:44.311851 9601 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785992227050689-9601-0/minicluster-data/master-0/wal/instance
I20260806 04:58:44.312718 9601 external_mini_cluster.cc:1428] Setting key a2f10fcc4f7673e5d6de26eea3a5356a
I20260806 04:58:44.320647 24426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:58:44.314733 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40800 (local address 127.9.96.126:40141)
0806 04:58:44.315704 (+ 971us) server_negotiation.cc:206] Beginning negotiation
0806 04:58:44.315714 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 04:58:44.315752 (+ 38us) server_negotiation.cc:407] Connection header received
0806 04:58:44.317221 (+ 1469us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:58:44.317241 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:58:44.317630 (+ 389us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:58:44.318009 (+ 379us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:58:44.319046 (+ 1037us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:58:44.319051 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:58:44.319062 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 04:58:44.319089 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:58:44.319171 (+ 82us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 04:58:44.319187 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:58:44.319464 (+ 277us) server_negotiation.cc:1035] Waiting for connection context
0806 04:58:44.319604 (+ 140us) server_negotiation.cc:299] Negotiation successful
0806 04:58:44.319826 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":552,"thread_start_us":349,"threads_started":1}
I20260806 04:58:44.320852 24421 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2f0a6eca06674cf0bae68ef844fa0c89: Bootstrap starting.
I20260806 04:58:44.324357 24421 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2f0a6eca06674cf0bae68ef844fa0c89: Neither blocks nor log segments found. Creating new log.
I20260806 04:58:44.325351 24421 log.cc:824] T 00000000000000000000000000000000 P 2f0a6eca06674cf0bae68ef844fa0c89: Log is configured to *not* fsync() on all Append() calls
I20260806 04:58:44.328310 24421 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2f0a6eca06674cf0bae68ef844fa0c89: No bootstrap required, opened a new log
I20260806 04:58:44.335125 24421 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2f0a6eca06674cf0bae68ef844fa0c89 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f0a6eca06674cf0bae68ef844fa0c89" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 40141 } }
I20260806 04:58:44.335558 24421 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2f0a6eca06674cf0bae68ef844fa0c89 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 04:58:44.335662 24421 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2f0a6eca06674cf0bae68ef844fa0c89 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2f0a6eca06674cf0bae68ef844fa0c89, State: Initialized, Role: FOLLOWER
I20260806 04:58:44.336231 24421 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2f0a6eca06674cf0bae68ef844fa0c89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f0a6eca06674cf0bae68ef844fa0c89" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 40141 } }
I20260806 04:58:44.336531 24421 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2f0a6eca06674cf0bae68ef844fa0c89 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 04:58:44.336680 24421 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2f0a6eca06674cf0bae68ef844fa0c89 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 04:58:44.336865 24421 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2f0a6eca06674cf0bae68ef844fa0c89 [term 0 FOLLOWER]: Advancing to term 1
I20260806 04:58:44.338968 24421 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2f0a6eca06674cf0bae68ef844fa0c89 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f0a6eca06674cf0bae68ef844fa0c89" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 40141 } }
I20260806 04:58:44.339542 24421 leader_election.cc:302] T 00000000000000000000000000000000 P 2f0a6eca06674cf0bae68ef844fa0c89 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2f0a6eca06674cf0bae68ef844fa0c89; no voters:
I20260806 04:58:44.340144 24421 leader_election.cc:288] T 00000000000000000000000000000000 P 2f0a6eca06674cf0bae68ef844fa0c89 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 04:58:44.340232 24429 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2f0a6eca06674cf0bae68ef844fa0c89 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 04:58:44.341076 24429 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2f0a6eca06674cf0bae68ef844fa0c89 [term 1 LEADER]: Becoming Leader. State: Replica: 2f0a6eca06674cf0bae68ef844fa0c89, State: Running, Role: LEADER
I20260806 04:58:44.341697 24429 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2f0a6eca06674cf0bae68ef844fa0c89 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f0a6eca06674cf0bae68ef844fa0c89" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 40141 } }
I20260806 04:58:44.342433 24421 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2f0a6eca06674cf0bae68ef844fa0c89 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 04:58:44.344794 24430 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2f0a6eca06674cf0bae68ef844fa0c89 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2f0a6eca06674cf0bae68ef844fa0c89" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f0a6eca06674cf0bae68ef844fa0c89" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 40141 } } }
I20260806 04:58:44.345268 24430 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2f0a6eca06674cf0bae68ef844fa0c89 [sys.catalog]: This master's current role is: LEADER
I20260806 04:58:44.345813 24431 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2f0a6eca06674cf0bae68ef844fa0c89 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2f0a6eca06674cf0bae68ef844fa0c89. Latest consensus state: current_term: 1 leader_uuid: "2f0a6eca06674cf0bae68ef844fa0c89" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f0a6eca06674cf0bae68ef844fa0c89" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 40141 } } }
I20260806 04:58:44.346202 24431 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2f0a6eca06674cf0bae68ef844fa0c89 [sys.catalog]: This master's current role is: LEADER
I20260806 04:58:44.348232 24439 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 04:58:44.353375 24439 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 04:58:44.362188 24439 catalog_manager.cc:1347] Generated new cluster ID: b0c65363633b4ba8a1db81bba4a3d262
I20260806 04:58:44.362344 24439 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 04:58:44.382468 24439 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 04:58:44.382874 24439 catalog_manager.cc:1504] Loading token signing keys...
I20260806 04:58:44.403649 24439 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2f0a6eca06674cf0bae68ef844fa0c89: Generated new TSK 0
I20260806 04:58:44.404703 24439 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 04:58:44.419325 9601 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
/tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785992227050689-9601-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785992227050689-9601-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785992227050689-9601-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785992227050689-9601-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.96.65:0
--local_ip_for_outbound_sockets=127.9.96.65
--webserver_interface=127.9.96.65
--webserver_port=0
--tserver_master_addrs=127.9.96.126:40141
--builtin_ntp_servers=127.9.96.84:33819
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260806 04:58:44.750489 24452 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 04:58:44.750738 24452 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 04:58:44.750821 24452 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 04:58:44.757957 24452 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 04:58:44.758046 24452 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 04:58:44.758169 24452 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.96.65
I20260806 04:58:44.767174 24452 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.96.84:33819
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785992227050689-9601-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785992227050689-9601-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.96.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785992227050689-9601-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.96.65
--webserver_port=0
--tserver_master_addrs=127.9.96.126:40141
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.96.65
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785992227050689-9601-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 03:56:12 UTC on ccycloud.quasar-guijbm.root.comops.site
build id 4569618
ASAN enabled
I20260806 04:58:44.768337 24452 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 04:58:44.769970 24452 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 04:58:44.783027 24458 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:58:44.782999 24463 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:58:44.782963 24459 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:58:44.784143 24452 server_base.cc:1034] running on GCE node
I20260806 04:58:44.784780 24452 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 04:58:44.785596 24452 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 04:58:44.786790 24452 hybrid_clock.cc:648] HybridClock initialized: now 1785992324786748 us; error 32 us; skew 500 ppm
I20260806 04:58:44.787132 24452 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 04:58:44.789366 24452 webserver.cc:460] Webserver started at http://127.9.96.65:39757/ using document root <none> and password file <none>
I20260806 04:58:44.790025 24452 fs_manager.cc:359] Metadata directory not provided
I20260806 04:58:44.790138 24452 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 04:58:44.790513 24452 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 04:58:44.795120 24452 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785992227050689-9601-0/minicluster-data/ts-0/data/instance:
uuid: "6fb9cd140e4744edbdc4222156c113e2"
format_stamp: "Formatted at 2026-08-06 04:58:44 on dist-test-slave-x4qh"
server_key: "fb786ca54b0e8f9060071f26b743ece3"
server_key_iv: "ba051b767711cbf5d57a2e5852a28dac"
server_key_version: "encryptionkey@0"
I20260806 04:58:44.795971 24452 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785992227050689-9601-0/minicluster-data/ts-0/wal/instance:
uuid: "6fb9cd140e4744edbdc4222156c113e2"
format_stamp: "Formatted at 2026-08-06 04:58:44 on dist-test-slave-x4qh"
server_key: "fb786ca54b0e8f9060071f26b743ece3"
server_key_iv: "ba051b767711cbf5d57a2e5852a28dac"
server_key_version: "encryptionkey@0"
I20260806 04:58:44.800683 24452 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260806 04:58:44.803831 24472 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 04:58:44.805068 24452 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260806 04:58:44.805248 24452 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785992227050689-9601-0/minicluster-data/ts-0/data,/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785992227050689-9601-0/minicluster-data/ts-0/wal
uuid: "6fb9cd140e4744edbdc4222156c113e2"
format_stamp: "Formatted at 2026-08-06 04:58:44 on dist-test-slave-x4qh"
server_key: "fb786ca54b0e8f9060071f26b743ece3"
server_key_iv: "ba051b767711cbf5d57a2e5852a28dac"
server_key_version: "encryptionkey@0"
I20260806 04:58:44.805459 24452 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785992227050689-9601-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785992227050689-9601-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785992227050689-9601-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 04:58:44.810849 24452 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 04:58:44.811615 24452 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 04:58:44.811892 24452 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 04:58:44.812752 24452 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 04:58:44.814039 24452 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 04:58:44.814149 24452 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 04:58:44.814275 24452 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 04:58:44.814380 24452 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 04:58:44.848291 24452 rpc_server.cc:307] RPC server started. Bound to: 127.9.96.65:45987
I20260806 04:58:44.848390 24627 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.96.65:45987 every 8 connection(s)
I20260806 04:58:44.849658 24452 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785992227050689-9601-0/minicluster-data/ts-0/data/info.pb
I20260806 04:58:44.855819 9601 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu as pid 24452
I20260806 04:58:44.856006 9601 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785992227050689-9601-0/minicluster-data/ts-0/wal/instance
I20260806 04:58:44.856849 9601 external_mini_cluster.cc:1428] Setting key d152468f6124a5ba4a2d350c9d69c6c9
I20260806 04:58:44.858037 24634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:58:44.852278 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.96.65:43765 (local address 127.9.96.126:40141)
0806 04:58:44.852754 (+ 476us) server_negotiation.cc:206] Beginning negotiation
0806 04:58:44.852759 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 04:58:44.853776 (+ 1017us) server_negotiation.cc:407] Connection header received
0806 04:58:44.855513 (+ 1737us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:58:44.855519 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:58:44.855593 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:58:44.855716 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:58:44.857247 (+ 1531us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:58:44.857253 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:58:44.857258 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 04:58:44.857283 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:58:44.857327 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 04:58:44.857338 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:58:44.857472 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0806 04:58:44.857670 (+ 198us) server_negotiation.cc:299] Negotiation successful
0806 04:58:44.857743 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":365,"thread_start_us":248,"threads_started":1}
I20260806 04:58:44.859014 24635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:58:44.852728 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:40141 (local address 127.9.96.65:43765)
0806 04:58:44.853623 (+ 895us) negotiation.cc:107] Waiting for socket to connect
0806 04:58:44.853665 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0806 04:58:44.855213 (+ 1548us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:58:44.855977 (+ 764us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:58:44.855989 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:58:44.856100 (+ 111us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:58:44.856895 (+ 795us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 04:58:44.856900 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:58:44.856952 (+ 52us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 04:58:44.857084 (+ 132us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 04:58:44.857439 (+ 355us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 04:58:44.857453 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 04:58:44.857460 (+ 7us) client_negotiation.cc:769] Sending connection context
0806 04:58:44.857750 (+ 290us) client_negotiation.cc:240] Negotiation successful
0806 04:58:44.857979 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":469,"thread_start_us":287,"threads_started":1}
I20260806 04:58:44.859875 9601 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
/tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785992227050689-9601-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785992227050689-9601-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785992227050689-9601-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785992227050689-9601-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.96.66:0
--local_ip_for_outbound_sockets=127.9.96.66
--webserver_interface=127.9.96.66
--webserver_port=0
--tserver_master_addrs=127.9.96.126:40141
--builtin_ntp_servers=127.9.96.84:33819
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260806 04:58:44.860559 24630 heartbeater.cc:347] Connected to a master server at 127.9.96.126:40141
I20260806 04:58:44.860898 24630 heartbeater.cc:464] Registering TS with master...
I20260806 04:58:44.861539 24630 heartbeater.cc:511] Master 127.9.96.126:40141 requested a full tablet report, sending...
I20260806 04:58:44.863708 24371 ts_manager.cc:194] Registered new tserver with Master: 6fb9cd140e4744edbdc4222156c113e2 (127.9.96.65:45987)
W20260806 04:58:45.197736 24636 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 04:58:45.198029 24636 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 04:58:45.198208 24636 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 04:58:45.206185 24636 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 04:58:45.206279 24636 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 04:58:45.206493 24636 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.96.66
I20260806 04:58:45.215871 24636 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.96.84:33819
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785992227050689-9601-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785992227050689-9601-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.96.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785992227050689-9601-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.96.66
--webserver_port=0
--tserver_master_addrs=127.9.96.126:40141
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.96.66
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785992227050689-9601-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 03:56:12 UTC on ccycloud.quasar-guijbm.root.comops.site
build id 4569618
ASAN enabled
I20260806 04:58:45.217036 24636 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 04:58:45.218782 24636 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 04:58:45.231690 24645 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:58:45.231769 24641 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:58:45.231808 24642 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:58:45.232812 24636 server_base.cc:1034] running on GCE node
I20260806 04:58:45.233371 24636 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 04:58:45.234234 24636 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 04:58:45.235471 24636 hybrid_clock.cc:648] HybridClock initialized: now 1785992325235414 us; error 51 us; skew 500 ppm
I20260806 04:58:45.235790 24636 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 04:58:45.237977 24636 webserver.cc:460] Webserver started at http://127.9.96.66:42195/ using document root <none> and password file <none>
I20260806 04:58:45.238620 24636 fs_manager.cc:359] Metadata directory not provided
I20260806 04:58:45.238737 24636 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 04:58:45.239100 24636 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 04:58:45.241833 24636 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785992227050689-9601-0/minicluster-data/ts-1/data/instance:
uuid: "8e5f80e1929942b5a7e866ffb7923286"
format_stamp: "Formatted at 2026-08-06 04:58:45 on dist-test-slave-x4qh"
server_key: "c133042d663726294588d52b4ebb885d"
server_key_iv: "328cde63241f9e3339a7392db7f71941"
server_key_version: "encryptionkey@0"
I20260806 04:58:45.242689 24636 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785992227050689-9601-0/minicluster-data/ts-1/wal/instance:
uuid: "8e5f80e1929942b5a7e866ffb7923286"
format_stamp: "Formatted at 2026-08-06 04:58:45 on dist-test-slave-x4qh"
server_key: "c133042d663726294588d52b4ebb885d"
server_key_iv: "328cde63241f9e3339a7392db7f71941"
server_key_version: "encryptionkey@0"
I20260806 04:58:45.247722 24636 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260806 04:58:45.251128 24656 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 04:58:45.252363 24636 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 04:58:45.252561 24636 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785992227050689-9601-0/minicluster-data/ts-1/data,/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785992227050689-9601-0/minicluster-data/ts-1/wal
uuid: "8e5f80e1929942b5a7e866ffb7923286"
format_stamp: "Formatted at 2026-08-06 04:58:45 on dist-test-slave-x4qh"
server_key: "c133042d663726294588d52b4ebb885d"
server_key_iv: "328cde63241f9e3339a7392db7f71941"
server_key_version: "encryptionkey@0"
I20260806 04:58:45.252789 24636 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785992227050689-9601-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785992227050689-9601-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785992227050689-9601-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 04:58:45.258231 24636 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 04:58:45.258963 24636 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 04:58:45.259241 24636 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 04:58:45.260309 24636 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 04:58:45.261615 24636 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 04:58:45.261744 24636 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 04:58:45.261871 24636 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 04:58:45.261958 24636 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 04:58:45.297115 24636 rpc_server.cc:307] RPC server started. Bound to: 127.9.96.66:39307
I20260806 04:58:45.297222 24815 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.96.66:39307 every 8 connection(s)
I20260806 04:58:45.298429 24636 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785992227050689-9601-0/minicluster-data/ts-1/data/info.pb
I20260806 04:58:45.306732 24634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:58:45.300993 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.96.66:39173 (local address 127.9.96.126:40141)
0806 04:58:45.301192 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0806 04:58:45.301196 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 04:58:45.302544 (+ 1348us) server_negotiation.cc:407] Connection header received
0806 04:58:45.304518 (+ 1974us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:58:45.304523 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:58:45.304596 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:58:45.304710 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:58:45.305992 (+ 1282us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:58:45.305998 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:58:45.306002 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 04:58:45.306021 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:58:45.306059 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 04:58:45.306071 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:58:45.306195 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0806 04:58:45.306392 (+ 197us) server_negotiation.cc:299] Negotiation successful
0806 04:58:45.306461 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260806 04:58:45.306792 9601 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu as pid 24636
I20260806 04:58:45.306965 9601 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785992227050689-9601-0/minicluster-data/ts-1/wal/instance
I20260806 04:58:45.307546 9601 external_mini_cluster.cc:1428] Setting key eb192e074c1d0c036fa2ff016491a277
I20260806 04:58:45.307718 24820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:58:45.301501 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:40141 (local address 127.9.96.66:39173)
0806 04:58:45.302391 (+ 890us) negotiation.cc:107] Waiting for socket to connect
0806 04:58:45.302435 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0806 04:58:45.304241 (+ 1806us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:58:45.304933 (+ 692us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:58:45.304941 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:58:45.305022 (+ 81us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:58:45.305706 (+ 684us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 04:58:45.305709 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:58:45.305745 (+ 36us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 04:58:45.305825 (+ 80us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 04:58:45.306169 (+ 344us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 04:58:45.306179 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 04:58:45.306187 (+ 8us) client_negotiation.cc:769] Sending connection context
0806 04:58:45.306474 (+ 287us) client_negotiation.cc:240] Negotiation successful
0806 04:58:45.306698 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":482,"thread_start_us":295,"threads_started":1}
I20260806 04:58:45.309357 24816 heartbeater.cc:347] Connected to a master server at 127.9.96.126:40141
I20260806 04:58:45.309669 24816 heartbeater.cc:464] Registering TS with master...
I20260806 04:58:45.310323 24816 heartbeater.cc:511] Master 127.9.96.126:40141 requested a full tablet report, sending...
I20260806 04:58:45.311043 9601 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
/tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785992227050689-9601-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785992227050689-9601-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785992227050689-9601-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785992227050689-9601-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.96.67:0
--local_ip_for_outbound_sockets=127.9.96.67
--webserver_interface=127.9.96.67
--webserver_port=0
--tserver_master_addrs=127.9.96.126:40141
--builtin_ntp_servers=127.9.96.84:33819
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260806 04:58:45.311961 24368 ts_manager.cc:194] Registered new tserver with Master: 8e5f80e1929942b5a7e866ffb7923286 (127.9.96.66:39307)
W20260806 04:58:45.640650 24821 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 04:58:45.641021 24821 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 04:58:45.641132 24821 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 04:58:45.648319 24821 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 04:58:45.648406 24821 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 04:58:45.648506 24821 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.96.67
I20260806 04:58:45.657451 24821 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.96.84:33819
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785992227050689-9601-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785992227050689-9601-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.96.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785992227050689-9601-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.96.67
--webserver_port=0
--tserver_master_addrs=127.9.96.126:40141
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.96.67
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785992227050689-9601-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 03:56:12 UTC on ccycloud.quasar-guijbm.root.comops.site
build id 4569618
ASAN enabled
I20260806 04:58:45.658648 24821 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 04:58:45.660322 24821 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 04:58:45.673494 24827 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:58:45.673570 24830 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:58:45.673527 24835 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:58:45.674424 24821 server_base.cc:1034] running on GCE node
I20260806 04:58:45.674954 24821 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 04:58:45.675864 24821 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 04:58:45.677088 24821 hybrid_clock.cc:648] HybridClock initialized: now 1785992325677054 us; error 25 us; skew 500 ppm
I20260806 04:58:45.677408 24821 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 04:58:45.679693 24821 webserver.cc:460] Webserver started at http://127.9.96.67:45343/ using document root <none> and password file <none>
I20260806 04:58:45.680372 24821 fs_manager.cc:359] Metadata directory not provided
I20260806 04:58:45.680485 24821 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 04:58:45.680830 24821 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 04:58:45.683790 24821 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785992227050689-9601-0/minicluster-data/ts-2/data/instance:
uuid: "c3d039ecc7e3440a94c329f7b843175e"
format_stamp: "Formatted at 2026-08-06 04:58:45 on dist-test-slave-x4qh"
server_key: "d833501ede042f5d20dfce0e2f453816"
server_key_iv: "67d0beb3c365cfdce46cc9c41f32d69f"
server_key_version: "encryptionkey@0"
I20260806 04:58:45.684578 24821 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785992227050689-9601-0/minicluster-data/ts-2/wal/instance:
uuid: "c3d039ecc7e3440a94c329f7b843175e"
format_stamp: "Formatted at 2026-08-06 04:58:45 on dist-test-slave-x4qh"
server_key: "d833501ede042f5d20dfce0e2f453816"
server_key_iv: "67d0beb3c365cfdce46cc9c41f32d69f"
server_key_version: "encryptionkey@0"
I20260806 04:58:45.689553 24821 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.003s
I20260806 04:58:45.696303 24846 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 04:58:45.697681 24821 fs_manager.cc:725] Time spent opening block manager: real 0.006s user 0.004s sys 0.000s
I20260806 04:58:45.697866 24821 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785992227050689-9601-0/minicluster-data/ts-2/data,/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785992227050689-9601-0/minicluster-data/ts-2/wal
uuid: "c3d039ecc7e3440a94c329f7b843175e"
format_stamp: "Formatted at 2026-08-06 04:58:45 on dist-test-slave-x4qh"
server_key: "d833501ede042f5d20dfce0e2f453816"
server_key_iv: "67d0beb3c365cfdce46cc9c41f32d69f"
server_key_version: "encryptionkey@0"
I20260806 04:58:45.698081 24821 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785992227050689-9601-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785992227050689-9601-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785992227050689-9601-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 04:58:45.703696 24821 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 04:58:45.704516 24821 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 04:58:45.704790 24821 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 04:58:45.705751 24821 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 04:58:45.707113 24821 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 04:58:45.707237 24821 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 04:58:45.707355 24821 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 04:58:45.707513 24821 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.001s
I20260806 04:58:45.740677 24821 rpc_server.cc:307] RPC server started. Bound to: 127.9.96.67:34841
I20260806 04:58:45.740777 25003 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.96.67:34841 every 8 connection(s)
I20260806 04:58:45.741961 24821 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785992227050689-9601-0/minicluster-data/ts-2/data/info.pb
I20260806 04:58:45.746838 9601 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu as pid 24821
I20260806 04:58:45.747030 9601 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785992227050689-9601-0/minicluster-data/ts-2/wal/instance
I20260806 04:58:45.747548 9601 external_mini_cluster.cc:1428] Setting key f2197a34f42e05770af5e424056f123c
I20260806 04:58:45.751183 24634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:58:45.744802 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.96.67:50689 (local address 127.9.96.126:40141)
0806 04:58:45.745011 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0806 04:58:45.745016 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 04:58:45.746339 (+ 1323us) server_negotiation.cc:407] Connection header received
0806 04:58:45.748177 (+ 1838us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:58:45.748182 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:58:45.748239 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:58:45.748334 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:58:45.749844 (+ 1510us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:58:45.749848 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:58:45.749852 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 04:58:45.749870 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:58:45.749904 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 04:58:45.749928 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:58:45.750039 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0806 04:58:45.750896 (+ 857us) server_negotiation.cc:299] Negotiation successful
0806 04:58:45.750948 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260806 04:58:45.752084 25006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:58:45.745311 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:40141 (local address 127.9.96.67:50689)
0806 04:58:45.746193 (+ 882us) negotiation.cc:107] Waiting for socket to connect
0806 04:58:45.746227 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0806 04:58:45.747915 (+ 1688us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:58:45.748573 (+ 658us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:58:45.748580 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:58:45.748661 (+ 81us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:58:45.749394 (+ 733us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 04:58:45.749397 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:58:45.749451 (+ 54us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 04:58:45.749558 (+ 107us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 04:58:45.750652 (+ 1094us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 04:58:45.750662 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 04:58:45.750671 (+ 9us) client_negotiation.cc:769] Sending connection context
0806 04:58:45.750958 (+ 287us) client_negotiation.cc:240] Negotiation successful
0806 04:58:45.751208 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":478,"thread_start_us":297,"threads_started":1}
I20260806 04:58:45.753341 25004 heartbeater.cc:347] Connected to a master server at 127.9.96.126:40141
I20260806 04:58:45.753598 25004 heartbeater.cc:464] Registering TS with master...
I20260806 04:58:45.754077 25004 heartbeater.cc:511] Master 127.9.96.126:40141 requested a full tablet report, sending...
I20260806 04:58:45.755354 24368 ts_manager.cc:194] Registered new tserver with Master: c3d039ecc7e3440a94c329f7b843175e (127.9.96.67:34841)
I20260806 04:58:45.762910 9601 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260806 04:58:45.769037 24634 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 04:58:45.767523 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40808 (local address 127.9.96.126:40141)
0806 04:58:45.767717 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0806 04:58:45.767722 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 04:58:45.768308 (+ 586us) server_negotiation.cc:407] Connection header received
0806 04:58:45.768426 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:58:45.768430 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:58:45.768480 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:58:45.768567 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:58:45.768847 (+ 280us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40808: BlockingRecv error: recv got EOF from 127.0.0.1:40808 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
I20260806 04:58:45.769902 9601 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu pid 24452
I20260806 04:58:45.773569 25016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:58:45.771015 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32984 (local address 127.9.96.65:45987)
0806 04:58:45.771588 (+ 573us) server_negotiation.cc:206] Beginning negotiation
0806 04:58:45.771594 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 04:58:45.771620 (+ 26us) server_negotiation.cc:407] Connection header received
0806 04:58:45.771680 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:58:45.771685 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:58:45.771859 (+ 174us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:58:45.771985 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:58:45.772857 (+ 872us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:58:45.772863 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:58:45.772873 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 04:58:45.772886 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:58:45.772927 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 04:58:45.772942 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:58:45.773061 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0806 04:58:45.773140 (+ 79us) server_negotiation.cc:299] Negotiation successful
0806 04:58:45.773262 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":400,"thread_start_us":252,"threads_started":1}
I20260806 04:58:45.774847 24623 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 04:58:45.856660 9601 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu with pid 24452
I20260806 04:58:45.879781 9601 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu pid 24636
I20260806 04:58:45.883450 25018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:58:45.880716 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50556 (local address 127.9.96.66:39307)
0806 04:58:45.881190 (+ 474us) server_negotiation.cc:206] Beginning negotiation
0806 04:58:45.881196 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 04:58:45.881221 (+ 25us) server_negotiation.cc:407] Connection header received
0806 04:58:45.881281 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:58:45.881286 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:58:45.881490 (+ 204us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:58:45.881698 (+ 208us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:58:45.882688 (+ 990us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:58:45.882695 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:58:45.882705 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 04:58:45.882718 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:58:45.882760 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 04:58:45.882777 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:58:45.882932 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0806 04:58:45.883009 (+ 77us) server_negotiation.cc:299] Negotiation successful
0806 04:58:45.883118 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":340,"thread_start_us":220,"threads_started":1}
I20260806 04:58:45.884672 24809 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 04:58:45.965715 9601 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu with pid 24636
I20260806 04:58:45.992398 9601 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu pid 24821
I20260806 04:58:45.996423 25022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:58:45.993547 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57690 (local address 127.9.96.67:34841)
0806 04:58:45.994187 (+ 640us) server_negotiation.cc:206] Beginning negotiation
0806 04:58:45.994194 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 04:58:45.994224 (+ 30us) server_negotiation.cc:407] Connection header received
0806 04:58:45.994290 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:58:45.994296 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:58:45.994507 (+ 211us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:58:45.994630 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:58:45.995724 (+ 1094us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:58:45.995730 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:58:45.995741 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 04:58:45.995766 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:58:45.995803 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 04:58:45.995816 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:58:45.995944 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0806 04:58:45.996082 (+ 138us) server_negotiation.cc:299] Negotiation successful
0806 04:58:45.996163 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":497,"thread_start_us":328,"threads_started":1}
I20260806 04:58:45.997685 24998 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 04:58:46.088646 9601 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu with pid 24821
I20260806 04:58:46.116953 9601 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu pid 24326
I20260806 04:58:46.118229 24415 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 04:58:46.217922 9601 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu with pid 24326
2026-08-06T04:58:46Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (2400 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785992227050689-9601-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 04:58:46 dist-test-slave-x4qh krb5kdc[25027](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 04:58:46 dist-test-slave-x4qh krb5kdc[25027](info): set up 2 sockets
Aug 06 04:58:46 dist-test-slave-x4qh krb5kdc[25027](info): commencing operation
krb5kdc: starting...
W20260806 04:58:47.505353 9601 mini_kdc.cc:121] Time spent starting KDC: real 1.242s user 0.000s sys 0.031s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785992227050689-9601-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785992227050689-9601-0/krb5kdc/test-user.keytab.
Aug 06 04:58:47 dist-test-slave-x4qh krb5kdc[25027](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992327, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T04:58:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T04:58:47Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.96.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.96.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.96.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.126.keytab.
Entry for principal kudu/127.9.96.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.126.keytab.
I20260806 04:58:47.758945 9601 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
/tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785992227050689-9601-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785992227050689-9601-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785992227050689-9601-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785992227050689-9601-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.96.126:44295
--webserver_interface=127.9.96.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.126.keytab
--principal=kudu/127.9.96.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.96.84:45611
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.96.126:44295
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785992227050689-9601-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785992227050689-9601-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785992227050689-9601-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 04:58:48.077487 25045 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 04:58:48.077739 25045 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 04:58:48.077903 25045 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 04:58:48.084666 25045 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 04:58:48.084789 25045 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 04:58:48.084897 25045 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 04:58:48.084996 25045 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 04:58:48.085083 25045 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 04:58:48.093887 25045 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.96.84:45611
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785992227050689-9601-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785992227050689-9601-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.96.126:44295
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.126.keytab
--principal=kudu/127.9.96.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.96.126:44295
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785992227050689-9601-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.96.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785992227050689-9601-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 03:56:12 UTC on ccycloud.quasar-guijbm.root.comops.site
build id 4569618
ASAN enabled
I20260806 04:58:48.095033 25045 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 04:58:48.096257 25045 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 04:58:48.107441 25053 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:58:48.107503 25052 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:58:48.107508 25055 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:58:48.108573 25045 server_base.cc:1034] running on GCE node
I20260806 04:58:48.109273 25045 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 04:58:48.110565 25045 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 04:58:48.111790 25045 hybrid_clock.cc:648] HybridClock initialized: now 1785992328111744 us; error 51 us; skew 500 ppm
I20260806 04:58:48.112126 25045 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 04:58:48 dist-test-slave-x4qh krb5kdc[25027](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992328, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.96.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 04:58:48.120230 25045 init.cc:377] Logged in from keytab as kudu/127.9.96.126@KRBTEST.COM (short username kudu)
I20260806 04:58:48.121575 25045 webserver.cc:460] Webserver started at http://127.9.96.126:32915/ using document root <none> and password file <none>
I20260806 04:58:48.122221 25045 fs_manager.cc:359] Metadata directory not provided
I20260806 04:58:48.122375 25045 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 04:58:48.122694 25045 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 04:58:48.125682 25045 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785992227050689-9601-0/minicluster-data/master-0/data/instance:
uuid: "a6642842c39f4f0faa1c228c95d13a1a"
format_stamp: "Formatted at 2026-08-06 04:58:48 on dist-test-slave-x4qh"
server_key: "f3fc7598d65e21fb390cd3339dc2316e"
server_key_iv: "d2b39dff3597f5f8214faca1869acd16"
server_key_version: "encryptionkey@0"
I20260806 04:58:48.126433 25045 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785992227050689-9601-0/minicluster-data/master-0/wal/instance:
uuid: "a6642842c39f4f0faa1c228c95d13a1a"
format_stamp: "Formatted at 2026-08-06 04:58:48 on dist-test-slave-x4qh"
server_key: "f3fc7598d65e21fb390cd3339dc2316e"
server_key_iv: "d2b39dff3597f5f8214faca1869acd16"
server_key_version: "encryptionkey@0"
I20260806 04:58:48.131317 25045 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260806 04:58:48.134516 25063 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 04:58:48.135787 25045 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 04:58:48.135978 25045 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785992227050689-9601-0/minicluster-data/master-0/data,/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785992227050689-9601-0/minicluster-data/master-0/wal
uuid: "a6642842c39f4f0faa1c228c95d13a1a"
format_stamp: "Formatted at 2026-08-06 04:58:48 on dist-test-slave-x4qh"
server_key: "f3fc7598d65e21fb390cd3339dc2316e"
server_key_iv: "d2b39dff3597f5f8214faca1869acd16"
server_key_version: "encryptionkey@0"
I20260806 04:58:48.136165 25045 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785992227050689-9601-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785992227050689-9601-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785992227050689-9601-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 04:58:48.158834 25045 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 04:58:48.163003 25045 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 04:58:48.163292 25045 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 04:58:48.181787 25045 rpc_server.cc:307] RPC server started. Bound to: 127.9.96.126:44295
I20260806 04:58:48.181826 25136 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.96.126:44295 every 8 connection(s)
I20260806 04:58:48.183063 25045 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785992227050689-9601-0/minicluster-data/master-0/data/info.pb
I20260806 04:58:48.186578 25138 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 04:58:48.191782 9601 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu as pid 25045
I20260806 04:58:48.191988 9601 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785992227050689-9601-0/minicluster-data/master-0/wal/instance
I20260806 04:58:48.192907 9601 external_mini_cluster.cc:1428] Setting key d9d65fb2fc740bd11326f919b7e81b44
I20260806 04:58:48.197520 25138 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a6642842c39f4f0faa1c228c95d13a1a: Bootstrap starting.
I20260806 04:58:48.201042 25138 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a6642842c39f4f0faa1c228c95d13a1a: Neither blocks nor log segments found. Creating new log.
I20260806 04:58:48.202221 25138 log.cc:824] T 00000000000000000000000000000000 P a6642842c39f4f0faa1c228c95d13a1a: Log is configured to *not* fsync() on all Append() calls
I20260806 04:58:48.205559 25138 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a6642842c39f4f0faa1c228c95d13a1a: No bootstrap required, opened a new log
Aug 06 04:58:48 dist-test-slave-x4qh krb5kdc[25027](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992327, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.96.126@KRBTEST.COM
I20260806 04:58:48.213815 25138 raft_consensus.cc:348] T 00000000000000000000000000000000 P a6642842c39f4f0faa1c228c95d13a1a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a6642842c39f4f0faa1c228c95d13a1a" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 44295 } }
I20260806 04:58:48.214179 25138 raft_consensus.cc:374] T 00000000000000000000000000000000 P a6642842c39f4f0faa1c228c95d13a1a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 04:58:48.214313 25138 raft_consensus.cc:729] T 00000000000000000000000000000000 P a6642842c39f4f0faa1c228c95d13a1a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a6642842c39f4f0faa1c228c95d13a1a, State: Initialized, Role: FOLLOWER
I20260806 04:58:48.214967 25138 consensus_queue.cc:260] T 00000000000000000000000000000000 P a6642842c39f4f0faa1c228c95d13a1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a6642842c39f4f0faa1c228c95d13a1a" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 44295 } }
I20260806 04:58:48.215232 25138 raft_consensus.cc:388] T 00000000000000000000000000000000 P a6642842c39f4f0faa1c228c95d13a1a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 04:58:48.215415 25138 raft_consensus.cc:482] T 00000000000000000000000000000000 P a6642842c39f4f0faa1c228c95d13a1a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 04:58:48.215648 25138 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a6642842c39f4f0faa1c228c95d13a1a [term 0 FOLLOWER]: Advancing to term 1
I20260806 04:58:48.216686 25144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:58:48.194981 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39790 (local address 127.9.96.126:44295)
0806 04:58:48.195844 (+ 863us) server_negotiation.cc:206] Beginning negotiation
0806 04:58:48.195854 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 04:58:48.195894 (+ 40us) server_negotiation.cc:407] Connection header received
0806 04:58:48.197379 (+ 1485us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:58:48.197398 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:58:48.197759 (+ 361us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:58:48.198081 (+ 322us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:58:48.200025 (+ 1944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:48.201207 (+ 1182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:48.202887 (+ 1680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:48.203896 (+ 1009us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:48.204036 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:58:48.208891 (+ 4855us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:58:48.208901 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:58:48.208918 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:58:48.208950 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:58:48.212467 (+ 3517us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:58:48.213195 (+ 728us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:58:48.213205 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:58:48.213218 (+ 13us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:58:48.213385 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:58:48.213769 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:58:48.213775 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:58:48.213778 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:58:48.214447 (+ 669us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:58:48.214793 (+ 346us) server_negotiation.cc:1035] Waiting for connection context
0806 04:58:48.215215 (+ 422us) server_negotiation.cc:299] Negotiation successful
0806 04:58:48.215674 (+ 459us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":653,"thread_start_us":441,"threads_started":1}
I20260806 04:58:48.218147 25138 raft_consensus.cc:504] T 00000000000000000000000000000000 P a6642842c39f4f0faa1c228c95d13a1a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a6642842c39f4f0faa1c228c95d13a1a" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 44295 } }
I20260806 04:58:48.218753 25138 leader_election.cc:302] T 00000000000000000000000000000000 P a6642842c39f4f0faa1c228c95d13a1a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a6642842c39f4f0faa1c228c95d13a1a; no voters:
I20260806 04:58:48.219548 25138 leader_election.cc:288] T 00000000000000000000000000000000 P a6642842c39f4f0faa1c228c95d13a1a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 04:58:48.219702 25147 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a6642842c39f4f0faa1c228c95d13a1a [term 1 FOLLOWER]: Leader election won for term 1
I20260806 04:58:48.220767 25147 raft_consensus.cc:686] T 00000000000000000000000000000000 P a6642842c39f4f0faa1c228c95d13a1a [term 1 LEADER]: Becoming Leader. State: Replica: a6642842c39f4f0faa1c228c95d13a1a, State: Running, Role: LEADER
I20260806 04:58:48.221267 25147 consensus_queue.cc:237] T 00000000000000000000000000000000 P a6642842c39f4f0faa1c228c95d13a1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a6642842c39f4f0faa1c228c95d13a1a" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 44295 } }
I20260806 04:58:48.221801 25138 sys_catalog.cc:564] T 00000000000000000000000000000000 P a6642842c39f4f0faa1c228c95d13a1a [sys.catalog]: configured and running, proceeding with master startup.
I20260806 04:58:48.224054 25148 sys_catalog.cc:455] T 00000000000000000000000000000000 P a6642842c39f4f0faa1c228c95d13a1a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a6642842c39f4f0faa1c228c95d13a1a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a6642842c39f4f0faa1c228c95d13a1a" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 44295 } } }
I20260806 04:58:48.224411 25148 sys_catalog.cc:458] T 00000000000000000000000000000000 P a6642842c39f4f0faa1c228c95d13a1a [sys.catalog]: This master's current role is: LEADER
I20260806 04:58:48.225286 25157 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 04:58:48.225098 25149 sys_catalog.cc:455] T 00000000000000000000000000000000 P a6642842c39f4f0faa1c228c95d13a1a [sys.catalog]: SysCatalogTable state changed. Reason: New leader a6642842c39f4f0faa1c228c95d13a1a. Latest consensus state: current_term: 1 leader_uuid: "a6642842c39f4f0faa1c228c95d13a1a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a6642842c39f4f0faa1c228c95d13a1a" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 44295 } } }
I20260806 04:58:48.227551 25149 sys_catalog.cc:458] T 00000000000000000000000000000000 P a6642842c39f4f0faa1c228c95d13a1a [sys.catalog]: This master's current role is: LEADER
I20260806 04:58:48.230203 25157 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 04:58:48.238240 25157 catalog_manager.cc:1347] Generated new cluster ID: 9610825e50e148e3b49a7d646155f292
I20260806 04:58:48.238399 25157 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 04:58:48.256897 25157 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 04:58:48.258580 25157 catalog_manager.cc:1504] Loading token signing keys...
I20260806 04:58:48.267895 25157 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a6642842c39f4f0faa1c228c95d13a1a: Generated new TSK 0
I20260806 04:58:48.268617 25157 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.9.96.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.96.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.96.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.65.keytab.
Entry for principal kudu/127.9.96.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.65.keytab.
I20260806 04:58:48.335044 9601 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
/tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785992227050689-9601-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785992227050689-9601-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785992227050689-9601-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785992227050689-9601-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.96.65:0
--local_ip_for_outbound_sockets=127.9.96.65
--webserver_interface=127.9.96.65
--webserver_port=0
--tserver_master_addrs=127.9.96.126:44295
--keytab_file=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.65.keytab
--principal=kudu/127.9.96.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.96.84:45611
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785992227050689-9601-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785992227050689-9601-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785992227050689-9601-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 04:58:48.668023 25177 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 04:58:48.668273 25177 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 04:58:48.668365 25177 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 04:58:48.675707 25177 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 04:58:48.675802 25177 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 04:58:48.675916 25177 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.96.65
I20260806 04:58:48.685186 25177 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.96.84:45611
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785992227050689-9601-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785992227050689-9601-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.65.keytab
--principal=kudu/127.9.96.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.96.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785992227050689-9601-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.96.65
--webserver_port=0
--tserver_master_addrs=127.9.96.126:44295
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.96.65
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785992227050689-9601-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 03:56:12 UTC on ccycloud.quasar-guijbm.root.comops.site
build id 4569618
ASAN enabled
I20260806 04:58:48.686295 25177 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 04:58:48.687546 25177 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 04:58:48.700222 25184 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:58:48.700263 25187 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:58:48.700294 25185 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:58:48.701273 25177 server_base.cc:1034] running on GCE node
I20260806 04:58:48.701823 25177 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 04:58:48.702732 25177 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 04:58:48.703998 25177 hybrid_clock.cc:648] HybridClock initialized: now 1785992328703935 us; error 46 us; skew 500 ppm
I20260806 04:58:48.704464 25177 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 04:58:48 dist-test-slave-x4qh krb5kdc[25027](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992328, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.96.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 04:58:48.707727 25177 init.cc:377] Logged in from keytab as kudu/127.9.96.65@KRBTEST.COM (short username kudu)
I20260806 04:58:48.709040 25177 webserver.cc:460] Webserver started at http://127.9.96.65:44537/ using document root <none> and password file <none>
I20260806 04:58:48.709635 25177 fs_manager.cc:359] Metadata directory not provided
I20260806 04:58:48.709735 25177 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 04:58:48.709967 25177 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 04:58:48.712803 25177 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785992227050689-9601-0/minicluster-data/ts-0/data/instance:
uuid: "cb97bdf0eac640deacd92387287e0f6a"
format_stamp: "Formatted at 2026-08-06 04:58:48 on dist-test-slave-x4qh"
server_key: "01ec95899de0627985a8e606b590a4e9"
server_key_iv: "9d26c0336df25150d7394a5c85f16650"
server_key_version: "encryptionkey@0"
I20260806 04:58:48.713510 25177 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785992227050689-9601-0/minicluster-data/ts-0/wal/instance:
uuid: "cb97bdf0eac640deacd92387287e0f6a"
format_stamp: "Formatted at 2026-08-06 04:58:48 on dist-test-slave-x4qh"
server_key: "01ec95899de0627985a8e606b590a4e9"
server_key_iv: "9d26c0336df25150d7394a5c85f16650"
server_key_version: "encryptionkey@0"
I20260806 04:58:48.718259 25177 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260806 04:58:48.721392 25195 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 04:58:48.722728 25177 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260806 04:58:48.722932 25177 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785992227050689-9601-0/minicluster-data/ts-0/data,/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785992227050689-9601-0/minicluster-data/ts-0/wal
uuid: "cb97bdf0eac640deacd92387287e0f6a"
format_stamp: "Formatted at 2026-08-06 04:58:48 on dist-test-slave-x4qh"
server_key: "01ec95899de0627985a8e606b590a4e9"
server_key_iv: "9d26c0336df25150d7394a5c85f16650"
server_key_version: "encryptionkey@0"
I20260806 04:58:48.723158 25177 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785992227050689-9601-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785992227050689-9601-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785992227050689-9601-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 04:58:48.737610 25177 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 04:58:48.741529 25177 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 04:58:48.741762 25177 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 04:58:48.742599 25177 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 04:58:48.743868 25177 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 04:58:48.743949 25177 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 04:58:48.744033 25177 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 04:58:48.744077 25177 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 04:58:48.777748 25177 rpc_server.cc:307] RPC server started. Bound to: 127.9.96.65:45757
I20260806 04:58:48.777834 25359 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.96.65:45757 every 8 connection(s)
I20260806 04:58:48.778987 25177 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785992227050689-9601-0/minicluster-data/ts-0/data/info.pb
I20260806 04:58:48.788616 9601 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu as pid 25177
I20260806 04:58:48.788815 9601 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785992227050689-9601-0/minicluster-data/ts-0/wal/instance
I20260806 04:58:48.789499 9601 external_mini_cluster.cc:1428] Setting key 2bc6bfa3b7ca4853af82cc2c9fba8ec3
Aug 06 04:58:48 dist-test-slave-x4qh krb5kdc[25027](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992328, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.96.65@KRBTEST.COM for kudu/127.9.96.126@KRBTEST.COM
I20260806 04:58:48.797878 25363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:58:48.781442 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.96.65:34545 (local address 127.9.96.126:44295)
0806 04:58:48.781905 (+ 463us) server_negotiation.cc:206] Beginning negotiation
0806 04:58:48.781909 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 04:58:48.782657 (+ 748us) server_negotiation.cc:407] Connection header received
0806 04:58:48.784558 (+ 1901us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:58:48.784565 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:58:48.784650 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:58:48.784771 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:58:48.786210 (+ 1439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:48.786934 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:48.788488 (+ 1554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:48.788974 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:48.789103 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:58:48.792495 (+ 3392us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:58:48.792499 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:58:48.792502 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:58:48.792530 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:58:48.794938 (+ 2408us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:58:48.795684 (+ 746us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:58:48.795689 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:58:48.795691 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:58:48.795802 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:58:48.796229 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:58:48.796234 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:58:48.796243 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:58:48.796502 (+ 259us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:58:48.796656 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0806 04:58:48.797313 (+ 657us) server_negotiation.cc:299] Negotiation successful
0806 04:58:48.797532 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":356,"thread_start_us":246,"threads_started":1}
I20260806 04:58:48.798707 25365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:58:48.781846 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:44295 (local address 127.9.96.65:34545)
0806 04:58:48.782515 (+ 669us) negotiation.cc:107] Waiting for socket to connect
0806 04:58:48.782549 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0806 04:58:48.784254 (+ 1705us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:58:48.785005 (+ 751us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:58:48.785015 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:58:48.785513 (+ 498us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:58:48.786041 (+ 528us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:58:48.786058 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:58:48.787102 (+ 1044us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:58:48.787107 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:58:48.788337 (+ 1230us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:58:48.788348 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:58:48.789200 (+ 852us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:58:48.789208 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:58:48.789455 (+ 247us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:58:48.790153 (+ 698us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:58:48.790173 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:58:48.792291 (+ 2118us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 04:58:48.795076 (+ 2785us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:58:48.795084 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:58:48.795097 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:58:48.795558 (+ 461us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:58:48.795913 (+ 355us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:58:48.795918 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:58:48.795921 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:58:48.796109 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:58:48.796633 (+ 524us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 04:58:48.796640 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 04:58:48.796931 (+ 291us) client_negotiation.cc:769] Sending connection context
0806 04:58:48.797172 (+ 241us) client_negotiation.cc:240] Negotiation successful
0806 04:58:48.797653 (+ 481us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":446,"thread_start_us":259,"threads_started":1}
I20260806 04:58:48.800197 25360 heartbeater.cc:347] Connected to a master server at 127.9.96.126:44295
I20260806 04:58:48.800478 25360 heartbeater.cc:464] Registering TS with master...
I20260806 04:58:48.801139 25360 heartbeater.cc:511] Master 127.9.96.126:44295 requested a full tablet report, sending...
I20260806 04:58:48.803087 25082 ts_manager.cc:194] Registered new tserver with Master: cb97bdf0eac640deacd92387287e0f6a (127.9.96.65:45757)
I20260806 04:58:48.804555 25082 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.96.65@KRBTEST.COM'} at 127.9.96.65:34545
WARNING: no policy specified for kudu/127.9.96.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.96.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.96.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.66.keytab.
Entry for principal kudu/127.9.96.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.66.keytab.
I20260806 04:58:48.846487 9601 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
/tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785992227050689-9601-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785992227050689-9601-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785992227050689-9601-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785992227050689-9601-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.96.66:0
--local_ip_for_outbound_sockets=127.9.96.66
--webserver_interface=127.9.96.66
--webserver_port=0
--tserver_master_addrs=127.9.96.126:44295
--keytab_file=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.66.keytab
--principal=kudu/127.9.96.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.96.84:45611
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785992227050689-9601-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785992227050689-9601-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785992227050689-9601-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 04:58:49.178921 25368 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 04:58:49.179165 25368 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 04:58:49.179263 25368 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 04:58:49.189085 25368 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 04:58:49.189214 25368 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 04:58:49.189401 25368 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.96.66
I20260806 04:58:49.204017 25368 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.96.84:45611
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785992227050689-9601-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785992227050689-9601-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.66.keytab
--principal=kudu/127.9.96.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.96.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785992227050689-9601-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.96.66
--webserver_port=0
--tserver_master_addrs=127.9.96.126:44295
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.96.66
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785992227050689-9601-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 03:56:12 UTC on ccycloud.quasar-guijbm.root.comops.site
build id 4569618
ASAN enabled
I20260806 04:58:49.205188 25368 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 04:58:49.206382 25368 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 04:58:49.219359 25379 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:58:49.219406 25375 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:58:49.219455 25377 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:58:49.220094 25368 server_base.cc:1034] running on GCE node
I20260806 04:58:49.220679 25368 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 04:58:49.221526 25368 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 04:58:49.222759 25368 hybrid_clock.cc:648] HybridClock initialized: now 1785992329222689 us; error 63 us; skew 500 ppm
I20260806 04:58:49.223073 25368 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 04:58:49 dist-test-slave-x4qh krb5kdc[25027](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992329, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.96.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 04:58:49.226255 25368 init.cc:377] Logged in from keytab as kudu/127.9.96.66@KRBTEST.COM (short username kudu)
I20260806 04:58:49.227825 25368 webserver.cc:460] Webserver started at http://127.9.96.66:34201/ using document root <none> and password file <none>
I20260806 04:58:49.228526 25368 fs_manager.cc:359] Metadata directory not provided
I20260806 04:58:49.228816 25368 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 04:58:49.229126 25368 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 04:58:49.232030 25368 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785992227050689-9601-0/minicluster-data/ts-1/data/instance:
uuid: "c01dc087864347b2a8e070a3504b5d14"
format_stamp: "Formatted at 2026-08-06 04:58:49 on dist-test-slave-x4qh"
server_key: "694a6f1639a5927b5ee1a37a8964e323"
server_key_iv: "e130ed0136596cef91bc65c4a2960d16"
server_key_version: "encryptionkey@0"
I20260806 04:58:49.232831 25368 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785992227050689-9601-0/minicluster-data/ts-1/wal/instance:
uuid: "c01dc087864347b2a8e070a3504b5d14"
format_stamp: "Formatted at 2026-08-06 04:58:49 on dist-test-slave-x4qh"
server_key: "694a6f1639a5927b5ee1a37a8964e323"
server_key_iv: "e130ed0136596cef91bc65c4a2960d16"
server_key_version: "encryptionkey@0"
I20260806 04:58:49.237692 25368 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260806 04:58:49.241297 25390 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 04:58:49.242581 25368 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 04:58:49.242769 25368 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785992227050689-9601-0/minicluster-data/ts-1/data,/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785992227050689-9601-0/minicluster-data/ts-1/wal
uuid: "c01dc087864347b2a8e070a3504b5d14"
format_stamp: "Formatted at 2026-08-06 04:58:49 on dist-test-slave-x4qh"
server_key: "694a6f1639a5927b5ee1a37a8964e323"
server_key_iv: "e130ed0136596cef91bc65c4a2960d16"
server_key_version: "encryptionkey@0"
I20260806 04:58:49.242977 25368 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785992227050689-9601-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785992227050689-9601-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785992227050689-9601-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 04:58:49.260622 25368 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 04:58:49.264437 25368 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 04:58:49.264767 25368 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 04:58:49.265654 25368 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 04:58:49.266901 25368 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 04:58:49.267002 25368 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 04:58:49.267153 25368 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 04:58:49.267234 25368 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 04:58:49.301096 25368 rpc_server.cc:307] RPC server started. Bound to: 127.9.96.66:36695
I20260806 04:58:49.301137 25558 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.96.66:36695 every 8 connection(s)
I20260806 04:58:49.302295 25368 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785992227050689-9601-0/minicluster-data/ts-1/data/info.pb
I20260806 04:58:49.312012 9601 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu as pid 25368
I20260806 04:58:49.312218 9601 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785992227050689-9601-0/minicluster-data/ts-1/wal/instance
I20260806 04:58:49.312700 9601 external_mini_cluster.cc:1428] Setting key 4360453c138fb85174cb8950a34ec909
Aug 06 04:58:49 dist-test-slave-x4qh krb5kdc[25027](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992329, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.96.66@KRBTEST.COM for kudu/127.9.96.126@KRBTEST.COM
I20260806 04:58:49.323467 25561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:58:49.304778 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.96.66:36851 (local address 127.9.96.126:44295)
0806 04:58:49.305221 (+ 443us) server_negotiation.cc:206] Beginning negotiation
0806 04:58:49.305226 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 04:58:49.306070 (+ 844us) server_negotiation.cc:407] Connection header received
0806 04:58:49.307933 (+ 1863us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:58:49.307938 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:58:49.308002 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:58:49.308084 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:58:49.309529 (+ 1445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:49.310304 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:49.311824 (+ 1520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:49.312332 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:49.312493 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:58:49.316178 (+ 3685us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:58:49.316187 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:58:49.316195 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:58:49.316240 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:58:49.319899 (+ 3659us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:58:49.320640 (+ 741us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:58:49.320648 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:58:49.320654 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:58:49.320831 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:58:49.321244 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:58:49.321250 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:58:49.321256 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:58:49.321658 (+ 402us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:58:49.321828 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0806 04:58:49.322649 (+ 821us) server_negotiation.cc:299] Negotiation successful
0806 04:58:49.322946 (+ 297us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":334,"thread_start_us":243,"threads_started":1}
I20260806 04:58:49.324178 25562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:58:49.305207 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:44295 (local address 127.9.96.66:36851)
0806 04:58:49.305922 (+ 715us) negotiation.cc:107] Waiting for socket to connect
0806 04:58:49.305958 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0806 04:58:49.307617 (+ 1659us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:58:49.308302 (+ 685us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:58:49.308309 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:58:49.308807 (+ 498us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:58:49.309380 (+ 573us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:58:49.309397 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:58:49.310425 (+ 1028us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:58:49.310429 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:58:49.311652 (+ 1223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:58:49.311664 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:58:49.312593 (+ 929us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:58:49.312600 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:58:49.312857 (+ 257us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:58:49.313550 (+ 693us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:58:49.313577 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:58:49.315929 (+ 2352us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 04:58:49.320047 (+ 4118us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:58:49.320054 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:58:49.320066 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:58:49.320508 (+ 442us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:58:49.320943 (+ 435us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:58:49.320946 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:58:49.320948 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:58:49.321128 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:58:49.321787 (+ 659us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 04:58:49.321792 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 04:58:49.322213 (+ 421us) client_negotiation.cc:769] Sending connection context
0806 04:58:49.322533 (+ 320us) client_negotiation.cc:240] Negotiation successful
0806 04:58:49.322982 (+ 449us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":489,"thread_start_us":300,"threads_started":1}
I20260806 04:58:49.325673 25559 heartbeater.cc:347] Connected to a master server at 127.9.96.126:44295
I20260806 04:58:49.325978 25559 heartbeater.cc:464] Registering TS with master...
I20260806 04:58:49.326828 25559 heartbeater.cc:511] Master 127.9.96.126:44295 requested a full tablet report, sending...
I20260806 04:58:49.328449 25082 ts_manager.cc:194] Registered new tserver with Master: c01dc087864347b2a8e070a3504b5d14 (127.9.96.66:36695)
I20260806 04:58:49.329284 25082 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.96.66@KRBTEST.COM'} at 127.9.96.66:36851
WARNING: no policy specified for kudu/127.9.96.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.96.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.96.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.67.keytab.
Entry for principal kudu/127.9.96.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.67.keytab.
I20260806 04:58:49.372452 9601 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
/tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785992227050689-9601-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785992227050689-9601-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785992227050689-9601-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785992227050689-9601-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.96.67:0
--local_ip_for_outbound_sockets=127.9.96.67
--webserver_interface=127.9.96.67
--webserver_port=0
--tserver_master_addrs=127.9.96.126:44295
--keytab_file=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.67.keytab
--principal=kudu/127.9.96.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.96.84:45611
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785992227050689-9601-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785992227050689-9601-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785992227050689-9601-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 04:58:49.696359 25567 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 04:58:49.696606 25567 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 04:58:49.696754 25567 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 04:58:49.703514 25567 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 04:58:49.703606 25567 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 04:58:49.703810 25567 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.96.67
I20260806 04:58:49.712780 25567 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.96.84:45611
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785992227050689-9601-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785992227050689-9601-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.67.keytab
--principal=kudu/127.9.96.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.96.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785992227050689-9601-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.96.67
--webserver_port=0
--tserver_master_addrs=127.9.96.126:44295
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.96.67
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785992227050689-9601-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 03:56:12 UTC on ccycloud.quasar-guijbm.root.comops.site
build id 4569618
ASAN enabled
I20260806 04:58:49.713943 25567 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 04:58:49.715157 25567 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 04:58:49.727558 25573 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:58:49.727583 25575 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:58:49.727818 25577 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:58:49.728933 25567 server_base.cc:1034] running on GCE node
I20260806 04:58:49.729527 25567 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 04:58:49.730321 25567 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 04:58:49.731547 25567 hybrid_clock.cc:648] HybridClock initialized: now 1785992329731501 us; error 61 us; skew 500 ppm
I20260806 04:58:49.731896 25567 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 04:58:49 dist-test-slave-x4qh krb5kdc[25027](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992329, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.96.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 04:58:49.735015 25567 init.cc:377] Logged in from keytab as kudu/127.9.96.67@KRBTEST.COM (short username kudu)
I20260806 04:58:49.736503 25567 webserver.cc:460] Webserver started at http://127.9.96.67:35771/ using document root <none> and password file <none>
I20260806 04:58:49.737159 25567 fs_manager.cc:359] Metadata directory not provided
I20260806 04:58:49.737305 25567 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 04:58:49.737605 25567 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 04:58:49.740727 25567 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785992227050689-9601-0/minicluster-data/ts-2/data/instance:
uuid: "6a88adec320744be9aea7dc0c0b1bdc0"
format_stamp: "Formatted at 2026-08-06 04:58:49 on dist-test-slave-x4qh"
server_key: "e33952105f75177ca987ecaec743ac96"
server_key_iv: "873ff1a4b9794c723f1c93f66298ba28"
server_key_version: "encryptionkey@0"
I20260806 04:58:49.741528 25567 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785992227050689-9601-0/minicluster-data/ts-2/wal/instance:
uuid: "6a88adec320744be9aea7dc0c0b1bdc0"
format_stamp: "Formatted at 2026-08-06 04:58:49 on dist-test-slave-x4qh"
server_key: "e33952105f75177ca987ecaec743ac96"
server_key_iv: "873ff1a4b9794c723f1c93f66298ba28"
server_key_version: "encryptionkey@0"
I20260806 04:58:49.746615 25567 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260806 04:58:49.749794 25586 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 04:58:49.751097 25567 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 04:58:49.751276 25567 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785992227050689-9601-0/minicluster-data/ts-2/data,/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785992227050689-9601-0/minicluster-data/ts-2/wal
uuid: "6a88adec320744be9aea7dc0c0b1bdc0"
format_stamp: "Formatted at 2026-08-06 04:58:49 on dist-test-slave-x4qh"
server_key: "e33952105f75177ca987ecaec743ac96"
server_key_iv: "873ff1a4b9794c723f1c93f66298ba28"
server_key_version: "encryptionkey@0"
I20260806 04:58:49.751540 25567 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785992227050689-9601-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785992227050689-9601-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785992227050689-9601-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 04:58:49.782874 25567 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 04:58:49.787129 25567 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 04:58:49.787490 25567 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 04:58:49.788414 25567 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 04:58:49.789693 25567 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 04:58:49.789808 25567 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 04:58:49.789968 25567 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 04:58:49.790042 25567 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 04:58:49.807219 25360 heartbeater.cc:503] Master 127.9.96.126:44295 was elected leader, sending a full tablet report...
I20260806 04:58:49.823575 25567 rpc_server.cc:307] RPC server started. Bound to: 127.9.96.67:44759
I20260806 04:58:49.823661 25746 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.96.67:44759 every 8 connection(s)
I20260806 04:58:49.824903 25567 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785992227050689-9601-0/minicluster-data/ts-2/data/info.pb
I20260806 04:58:49.835038 9601 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu as pid 25567
I20260806 04:58:49.835210 9601 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785992227050689-9601-0/minicluster-data/ts-2/wal/instance
I20260806 04:58:49.835834 9601 external_mini_cluster.cc:1428] Setting key c913783a755f3d5683adc684ed6986bc
Aug 06 04:58:49 dist-test-slave-x4qh krb5kdc[25027](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992329, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.96.67@KRBTEST.COM for kudu/127.9.96.126@KRBTEST.COM
I20260806 04:58:49.845541 25749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:58:49.827534 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.96.67:41879 (local address 127.9.96.126:44295)
0806 04:58:49.828042 (+ 508us) server_negotiation.cc:206] Beginning negotiation
0806 04:58:49.828046 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 04:58:49.828960 (+ 914us) server_negotiation.cc:407] Connection header received
0806 04:58:49.830758 (+ 1798us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:58:49.830773 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:58:49.830904 (+ 131us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:58:49.831037 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:58:49.832517 (+ 1480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:49.833350 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:49.834900 (+ 1550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:49.835451 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:49.835551 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:58:49.840020 (+ 4469us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:58:49.840025 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:58:49.840028 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:58:49.840059 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:58:49.842505 (+ 2446us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:58:49.843201 (+ 696us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:58:49.843208 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:58:49.843217 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:58:49.843335 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:58:49.843820 (+ 485us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:58:49.843825 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:58:49.843830 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:58:49.844098 (+ 268us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:58:49.844270 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0806 04:58:49.844947 (+ 677us) server_negotiation.cc:299] Negotiation successful
0806 04:58:49.845167 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":399,"thread_start_us":285,"threads_started":1}
I20260806 04:58:49.846500 25750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:58:49.828120 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:44295 (local address 127.9.96.67:41879)
0806 04:58:49.828822 (+ 702us) negotiation.cc:107] Waiting for socket to connect
0806 04:58:49.828857 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0806 04:58:49.830481 (+ 1624us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:58:49.831291 (+ 810us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:58:49.831298 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:58:49.831823 (+ 525us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:58:49.832350 (+ 527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:58:49.832365 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:58:49.833514 (+ 1149us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:58:49.833518 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:58:49.834751 (+ 1233us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:58:49.834767 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:58:49.835677 (+ 910us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:58:49.835684 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:58:49.835978 (+ 294us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:58:49.836778 (+ 800us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:58:49.836807 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:58:49.839782 (+ 2975us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 04:58:49.842637 (+ 2855us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:58:49.842646 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:58:49.842660 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:58:49.843072 (+ 412us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:58:49.843484 (+ 412us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:58:49.843489 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:58:49.843493 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:58:49.843688 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:58:49.844239 (+ 551us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 04:58:49.844246 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 04:58:49.844588 (+ 342us) client_negotiation.cc:769] Sending connection context
0806 04:58:49.844840 (+ 252us) client_negotiation.cc:240] Negotiation successful
0806 04:58:49.845324 (+ 484us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":480,"thread_start_us":304,"threads_started":1}
I20260806 04:58:49.847827 25747 heartbeater.cc:347] Connected to a master server at 127.9.96.126:44295
I20260806 04:58:49.848083 25747 heartbeater.cc:464] Registering TS with master...
I20260806 04:58:49.848726 25747 heartbeater.cc:511] Master 127.9.96.126:44295 requested a full tablet report, sending...
I20260806 04:58:49.850124 25082 ts_manager.cc:194] Registered new tserver with Master: 6a88adec320744be9aea7dc0c0b1bdc0 (127.9.96.67:44759)
I20260806 04:58:49.851037 25082 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.96.67@KRBTEST.COM'} at 127.9.96.67:41879
I20260806 04:58:49.851500 9601 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 04:58:49.868795 25749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:58:49.856612 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39802 (local address 127.9.96.126:44295)
0806 04:58:49.856797 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0806 04:58:49.856805 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 04:58:49.857202 (+ 397us) server_negotiation.cc:407] Connection header received
0806 04:58:49.857425 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:58:49.857447 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:58:49.857501 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:58:49.857598 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:58:49.859058 (+ 1460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:49.859801 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:49.861068 (+ 1267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:49.861493 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:49.861639 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:58:49.863807 (+ 2168us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:58:49.863812 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:58:49.863815 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:58:49.863843 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:58:49.866124 (+ 2281us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:58:49.866742 (+ 618us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:58:49.866749 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:58:49.866752 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:58:49.866877 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:58:49.867236 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:58:49.867241 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:58:49.867244 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:58:49.867613 (+ 369us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:58:49.867876 (+ 263us) server_negotiation.cc:1035] Waiting for connection context
0806 04:58:49.868193 (+ 317us) server_negotiation.cc:299] Negotiation successful
0806 04:58:49.868401 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260806 04:58:49.872646 25082 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39802:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 04:58:49.879724 25082 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 04:58:49.911164 25761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:58:49.891413 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.65:45757 (local address 127.0.0.1:57570)
0806 04:58:49.903333 (+ 11920us) negotiation.cc:107] Waiting for socket to connect
0806 04:58:49.903351 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 04:58:49.903526 (+ 175us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:58:49.904468 (+ 942us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:58:49.904474 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:58:49.904503 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 04:58:49.904971 (+ 468us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:58:49.904987 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:58:49.906840 (+ 1853us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:58:49.906850 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:58:49.908726 (+ 1876us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:58:49.908738 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:58:49.910296 (+ 1558us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:58:49.910311 (+ 15us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:58:49.910509 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:58:49.910567 (+ 58us) client_negotiation.cc:769] Sending connection context
0806 04:58:49.910729 (+ 162us) client_negotiation.cc:240] Negotiation successful
0806 04:58:49.910897 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11764,"thread_start_us":12014,"threads_started":1}
I20260806 04:58:49.912074 25766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:58:49.892004 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57570 (local address 127.9.96.65:45757)
0806 04:58:49.902867 (+ 10863us) server_negotiation.cc:206] Beginning negotiation
0806 04:58:49.902874 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 04:58:49.903666 (+ 792us) server_negotiation.cc:407] Connection header received
0806 04:58:49.903770 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:58:49.903777 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:58:49.903997 (+ 220us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:58:49.904197 (+ 200us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 04:58:49.905190 (+ 993us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:49.906686 (+ 1496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:49.908888 (+ 2202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:49.910082 (+ 1194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:49.910235 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:58:49.910528 (+ 293us) server_negotiation.cc:1035] Waiting for connection context
0806 04:58:49.911590 (+ 1062us) server_negotiation.cc:299] Negotiation successful
0806 04:58:49.911760 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10705,"thread_start_us":10557,"threads_started":1}
I20260806 04:58:49.912160 25765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:58:49.891284 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.67:44759 (local address 127.0.0.1:60882)
0806 04:58:49.901191 (+ 9907us) negotiation.cc:107] Waiting for socket to connect
0806 04:58:49.901240 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0806 04:58:49.901431 (+ 191us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:58:49.903536 (+ 2105us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:58:49.903544 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:58:49.903576 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 04:58:49.904071 (+ 495us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:58:49.904086 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:58:49.906211 (+ 2125us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:58:49.906216 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:58:49.908973 (+ 2757us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:58:49.908988 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:58:49.911358 (+ 2370us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:58:49.911364 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:58:49.911539 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:58:49.911570 (+ 31us) client_negotiation.cc:769] Sending connection context
0806 04:58:49.911648 (+ 78us) client_negotiation.cc:240] Negotiation successful
0806 04:58:49.911762 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9605,"mutex_wait_us":148,"thread_start_us":11732,"threads_started":1}
I20260806 04:58:49.912328 25767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:58:49.892509 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41892 (local address 127.9.96.66:36695)
0806 04:58:49.902080 (+ 9571us) server_negotiation.cc:206] Beginning negotiation
0806 04:58:49.902093 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 04:58:49.903820 (+ 1727us) server_negotiation.cc:407] Connection header received
0806 04:58:49.904162 (+ 342us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:58:49.904169 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:58:49.904375 (+ 206us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:58:49.904549 (+ 174us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 04:58:49.905465 (+ 916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:49.906751 (+ 1286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:49.909465 (+ 2714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:49.910936 (+ 1471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:49.911034 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:58:49.911223 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0806 04:58:49.911827 (+ 604us) server_negotiation.cc:299] Negotiation successful
0806 04:58:49.912002 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9363,"thread_start_us":9247,"threads_started":1}
I20260806 04:58:49.912433 25764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:58:49.891511 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60882 (local address 127.9.96.67:44759)
0806 04:58:49.902550 (+ 11039us) server_negotiation.cc:206] Beginning negotiation
0806 04:58:49.902561 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 04:58:49.902597 (+ 36us) server_negotiation.cc:407] Connection header received
0806 04:58:49.902700 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:58:49.902710 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:58:49.902983 (+ 273us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:58:49.903193 (+ 210us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 04:58:49.904299 (+ 1106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:49.906056 (+ 1757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:49.909178 (+ 3122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:49.911139 (+ 1961us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:49.911275 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:58:49.911525 (+ 250us) server_negotiation.cc:1035] Waiting for connection context
0806 04:58:49.911966 (+ 441us) server_negotiation.cc:299] Negotiation successful
0806 04:58:49.912117 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10856,"thread_start_us":10689,"threads_started":1}
I20260806 04:58:49.913091 25762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:58:49.891894 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.66:36695 (local address 127.0.0.1:41892)
0806 04:58:49.903707 (+ 11813us) negotiation.cc:107] Waiting for socket to connect
0806 04:58:49.903723 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 04:58:49.903939 (+ 216us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:58:49.904773 (+ 834us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:58:49.904776 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:58:49.904795 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 04:58:49.905271 (+ 476us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:58:49.905283 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:58:49.906955 (+ 1672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:58:49.906966 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:58:49.909277 (+ 2311us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:58:49.909289 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:58:49.911441 (+ 2152us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:58:49.911448 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:58:49.911623 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:58:49.911656 (+ 33us) client_negotiation.cc:769] Sending connection context
0806 04:58:49.912609 (+ 953us) client_negotiation.cc:240] Negotiation successful
0806 04:58:49.912743 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11670,"thread_start_us":8853,"threads_started":1}
I20260806 04:58:49.916275 25652 tablet_service.cc:1467] Processing CreateTablet for tablet 9b9718d6d2454967bb1ea16b087884b4 (DEFAULT_TABLE table=test-table [id=35e9e8d83608429c938b740bcc1e5c0a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 04:58:49.916317 25462 tablet_service.cc:1467] Processing CreateTablet for tablet 9b9718d6d2454967bb1ea16b087884b4 (DEFAULT_TABLE table=test-table [id=35e9e8d83608429c938b740bcc1e5c0a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 04:58:49.916306 25267 tablet_service.cc:1467] Processing CreateTablet for tablet 9b9718d6d2454967bb1ea16b087884b4 (DEFAULT_TABLE table=test-table [id=35e9e8d83608429c938b740bcc1e5c0a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 04:58:49.917537 25652 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9b9718d6d2454967bb1ea16b087884b4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 04:58:49.917647 25267 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9b9718d6d2454967bb1ea16b087884b4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 04:58:49.917672 25462 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9b9718d6d2454967bb1ea16b087884b4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 04:58:49.930301 25768 tablet_bootstrap.cc:492] T 9b9718d6d2454967bb1ea16b087884b4 P c01dc087864347b2a8e070a3504b5d14: Bootstrap starting.
I20260806 04:58:49.932272 25769 tablet_bootstrap.cc:492] T 9b9718d6d2454967bb1ea16b087884b4 P 6a88adec320744be9aea7dc0c0b1bdc0: Bootstrap starting.
I20260806 04:58:49.933820 25772 tablet_bootstrap.cc:492] T 9b9718d6d2454967bb1ea16b087884b4 P cb97bdf0eac640deacd92387287e0f6a: Bootstrap starting.
I20260806 04:58:49.935770 25769 tablet_bootstrap.cc:654] T 9b9718d6d2454967bb1ea16b087884b4 P 6a88adec320744be9aea7dc0c0b1bdc0: Neither blocks nor log segments found. Creating new log.
I20260806 04:58:49.935770 25768 tablet_bootstrap.cc:654] T 9b9718d6d2454967bb1ea16b087884b4 P c01dc087864347b2a8e070a3504b5d14: Neither blocks nor log segments found. Creating new log.
I20260806 04:58:49.936717 25772 tablet_bootstrap.cc:654] T 9b9718d6d2454967bb1ea16b087884b4 P cb97bdf0eac640deacd92387287e0f6a: Neither blocks nor log segments found. Creating new log.
I20260806 04:58:49.937107 25769 log.cc:824] T 9b9718d6d2454967bb1ea16b087884b4 P 6a88adec320744be9aea7dc0c0b1bdc0: Log is configured to *not* fsync() on all Append() calls
I20260806 04:58:49.937107 25768 log.cc:824] T 9b9718d6d2454967bb1ea16b087884b4 P c01dc087864347b2a8e070a3504b5d14: Log is configured to *not* fsync() on all Append() calls
I20260806 04:58:49.937786 25772 log.cc:824] T 9b9718d6d2454967bb1ea16b087884b4 P cb97bdf0eac640deacd92387287e0f6a: Log is configured to *not* fsync() on all Append() calls
I20260806 04:58:49.940374 25769 tablet_bootstrap.cc:492] T 9b9718d6d2454967bb1ea16b087884b4 P 6a88adec320744be9aea7dc0c0b1bdc0: No bootstrap required, opened a new log
I20260806 04:58:49.940366 25768 tablet_bootstrap.cc:492] T 9b9718d6d2454967bb1ea16b087884b4 P c01dc087864347b2a8e070a3504b5d14: No bootstrap required, opened a new log
I20260806 04:58:49.940567 25772 tablet_bootstrap.cc:492] T 9b9718d6d2454967bb1ea16b087884b4 P cb97bdf0eac640deacd92387287e0f6a: No bootstrap required, opened a new log
I20260806 04:58:49.940653 25768 ts_tablet_manager.cc:1397] T 9b9718d6d2454967bb1ea16b087884b4 P c01dc087864347b2a8e070a3504b5d14: Time spent bootstrapping tablet: real 0.011s user 0.004s sys 0.003s
I20260806 04:58:49.940653 25769 ts_tablet_manager.cc:1397] T 9b9718d6d2454967bb1ea16b087884b4 P 6a88adec320744be9aea7dc0c0b1bdc0: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260806 04:58:49.940891 25772 ts_tablet_manager.cc:1397] T 9b9718d6d2454967bb1ea16b087884b4 P cb97bdf0eac640deacd92387287e0f6a: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.005s
I20260806 04:58:49.948501 25772 raft_consensus.cc:348] T 9b9718d6d2454967bb1ea16b087884b4 P cb97bdf0eac640deacd92387287e0f6a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a88adec320744be9aea7dc0c0b1bdc0" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 44759 } } peers { permanent_uuid: "cb97bdf0eac640deacd92387287e0f6a" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 45757 } } peers { permanent_uuid: "c01dc087864347b2a8e070a3504b5d14" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 36695 } }
I20260806 04:58:49.948853 25772 raft_consensus.cc:374] T 9b9718d6d2454967bb1ea16b087884b4 P cb97bdf0eac640deacd92387287e0f6a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 04:58:49.948953 25772 raft_consensus.cc:729] T 9b9718d6d2454967bb1ea16b087884b4 P cb97bdf0eac640deacd92387287e0f6a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb97bdf0eac640deacd92387287e0f6a, State: Initialized, Role: FOLLOWER
I20260806 04:58:49.949568 25772 consensus_queue.cc:260] T 9b9718d6d2454967bb1ea16b087884b4 P cb97bdf0eac640deacd92387287e0f6a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a88adec320744be9aea7dc0c0b1bdc0" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 44759 } } peers { permanent_uuid: "cb97bdf0eac640deacd92387287e0f6a" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 45757 } } peers { permanent_uuid: "c01dc087864347b2a8e070a3504b5d14" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 36695 } }
I20260806 04:58:49.951215 25772 ts_tablet_manager.cc:1428] T 9b9718d6d2454967bb1ea16b087884b4 P cb97bdf0eac640deacd92387287e0f6a: Time spent starting tablet: real 0.010s user 0.009s sys 0.000s
I20260806 04:58:49.951401 25769 raft_consensus.cc:348] T 9b9718d6d2454967bb1ea16b087884b4 P 6a88adec320744be9aea7dc0c0b1bdc0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a88adec320744be9aea7dc0c0b1bdc0" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 44759 } } peers { permanent_uuid: "cb97bdf0eac640deacd92387287e0f6a" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 45757 } } peers { permanent_uuid: "c01dc087864347b2a8e070a3504b5d14" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 36695 } }
I20260806 04:58:49.951372 25768 raft_consensus.cc:348] T 9b9718d6d2454967bb1ea16b087884b4 P c01dc087864347b2a8e070a3504b5d14 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a88adec320744be9aea7dc0c0b1bdc0" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 44759 } } peers { permanent_uuid: "cb97bdf0eac640deacd92387287e0f6a" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 45757 } } peers { permanent_uuid: "c01dc087864347b2a8e070a3504b5d14" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 36695 } }
I20260806 04:58:49.951797 25768 raft_consensus.cc:374] T 9b9718d6d2454967bb1ea16b087884b4 P c01dc087864347b2a8e070a3504b5d14 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 04:58:49.951794 25769 raft_consensus.cc:374] T 9b9718d6d2454967bb1ea16b087884b4 P 6a88adec320744be9aea7dc0c0b1bdc0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 04:58:49.951891 25769 raft_consensus.cc:729] T 9b9718d6d2454967bb1ea16b087884b4 P 6a88adec320744be9aea7dc0c0b1bdc0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a88adec320744be9aea7dc0c0b1bdc0, State: Initialized, Role: FOLLOWER
I20260806 04:58:49.951903 25768 raft_consensus.cc:729] T 9b9718d6d2454967bb1ea16b087884b4 P c01dc087864347b2a8e070a3504b5d14 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c01dc087864347b2a8e070a3504b5d14, State: Initialized, Role: FOLLOWER
I20260806 04:58:49.952497 25769 consensus_queue.cc:260] T 9b9718d6d2454967bb1ea16b087884b4 P 6a88adec320744be9aea7dc0c0b1bdc0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a88adec320744be9aea7dc0c0b1bdc0" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 44759 } } peers { permanent_uuid: "cb97bdf0eac640deacd92387287e0f6a" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 45757 } } peers { permanent_uuid: "c01dc087864347b2a8e070a3504b5d14" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 36695 } }
I20260806 04:58:49.952457 25768 consensus_queue.cc:260] T 9b9718d6d2454967bb1ea16b087884b4 P c01dc087864347b2a8e070a3504b5d14 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a88adec320744be9aea7dc0c0b1bdc0" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 44759 } } peers { permanent_uuid: "cb97bdf0eac640deacd92387287e0f6a" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 45757 } } peers { permanent_uuid: "c01dc087864347b2a8e070a3504b5d14" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 36695 } }
I20260806 04:58:49.953594 25559 heartbeater.cc:503] Master 127.9.96.126:44295 was elected leader, sending a full tablet report...
I20260806 04:58:49.954397 25768 ts_tablet_manager.cc:1428] T 9b9718d6d2454967bb1ea16b087884b4 P c01dc087864347b2a8e070a3504b5d14: Time spent starting tablet: real 0.014s user 0.005s sys 0.007s
I20260806 04:58:49.955603 25747 heartbeater.cc:503] Master 127.9.96.126:44295 was elected leader, sending a full tablet report...
I20260806 04:58:49.956305 25769 ts_tablet_manager.cc:1428] T 9b9718d6d2454967bb1ea16b087884b4 P 6a88adec320744be9aea7dc0c0b1bdc0: Time spent starting tablet: real 0.016s user 0.012s sys 0.004s
I20260806 04:58:49.961778 25779 raft_consensus.cc:482] T 9b9718d6d2454967bb1ea16b087884b4 P 6a88adec320744be9aea7dc0c0b1bdc0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 04:58:49.962009 25779 raft_consensus.cc:504] T 9b9718d6d2454967bb1ea16b087884b4 P 6a88adec320744be9aea7dc0c0b1bdc0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a88adec320744be9aea7dc0c0b1bdc0" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 44759 } } peers { permanent_uuid: "cb97bdf0eac640deacd92387287e0f6a" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 45757 } } peers { permanent_uuid: "c01dc087864347b2a8e070a3504b5d14" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 36695 } }
I20260806 04:58:49.963455 25779 leader_election.cc:288] T 9b9718d6d2454967bb1ea16b087884b4 P 6a88adec320744be9aea7dc0c0b1bdc0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cb97bdf0eac640deacd92387287e0f6a (127.9.96.65:45757), c01dc087864347b2a8e070a3504b5d14 (127.9.96.66:36695)
I20260806 04:58:49.969717 25750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:58:49.963869 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.65:45757 (local address 127.9.96.67:43875)
0806 04:58:49.964068 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0806 04:58:49.964088 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0806 04:58:49.964204 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:58:49.964625 (+ 421us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:58:49.964629 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:58:49.964657 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 04:58:49.964877 (+ 220us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:58:49.964887 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:58:49.966221 (+ 1334us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:58:49.966224 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:58:49.967982 (+ 1758us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:58:49.967998 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:58:49.969205 (+ 1207us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:58:49.969209 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:58:49.969341 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:58:49.969365 (+ 24us) client_negotiation.cc:769] Sending connection context
0806 04:58:49.969438 (+ 73us) client_negotiation.cc:240] Negotiation successful
0806 04:58:49.969523 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":52,"mutex_wait_us":34}
I20260806 04:58:49.969848 25766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:58:49.963927 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.96.67:43875 (local address 127.9.96.65:45757)
0806 04:58:49.964111 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0806 04:58:49.964118 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 04:58:49.964137 (+ 19us) server_negotiation.cc:407] Connection header received
0806 04:58:49.964344 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:58:49.964348 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:58:49.964460 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:58:49.964556 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 04:58:49.965087 (+ 531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:49.966096 (+ 1009us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:49.968131 (+ 2035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:49.969051 (+ 920us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:49.969186 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:58:49.969289 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0806 04:58:49.969483 (+ 194us) server_negotiation.cc:299] Negotiation successful
0806 04:58:49.969613 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260806 04:58:49.971053 25767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:58:49.964003 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.96.67:39151 (local address 127.9.96.66:36695)
0806 04:58:49.964177 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0806 04:58:49.964181 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 04:58:49.965306 (+ 1125us) server_negotiation.cc:407] Connection header received
0806 04:58:49.965469 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:58:49.965473 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:58:49.965544 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:58:49.965651 (+ 107us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 04:58:49.966198 (+ 547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:49.966890 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:49.968917 (+ 2027us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:49.970139 (+ 1222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:49.970282 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:58:49.970376 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0806 04:58:49.970747 (+ 371us) server_negotiation.cc:299] Negotiation successful
0806 04:58:49.970830 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260806 04:58:49.970789 25293 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9b9718d6d2454967bb1ea16b087884b4" candidate_uuid: "6a88adec320744be9aea7dc0c0b1bdc0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb97bdf0eac640deacd92387287e0f6a" is_pre_election: true
I20260806 04:58:49.971306 25784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:58:49.963890 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.66:36695 (local address 127.9.96.67:39151)
0806 04:58:49.965195 (+ 1305us) negotiation.cc:107] Waiting for socket to connect
0806 04:58:49.965214 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0806 04:58:49.965347 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:58:49.965686 (+ 339us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:58:49.965690 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:58:49.965716 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 04:58:49.966041 (+ 325us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:58:49.966047 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:58:49.967040 (+ 993us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:58:49.967045 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:58:49.968739 (+ 1694us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:58:49.968750 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:58:49.970366 (+ 1616us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:58:49.970374 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:58:49.970561 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:58:49.970596 (+ 35us) client_negotiation.cc:769] Sending connection context
0806 04:58:49.970708 (+ 112us) client_negotiation.cc:240] Negotiation successful
0806 04:58:49.970834 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1181,"mutex_wait_us":425,"thread_start_us":612,"threads_started":1}
I20260806 04:58:49.971377 25293 raft_consensus.cc:2454] T 9b9718d6d2454967bb1ea16b087884b4 P cb97bdf0eac640deacd92387287e0f6a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a88adec320744be9aea7dc0c0b1bdc0 in term 0.
I20260806 04:58:49.972193 25591 leader_election.cc:302] T 9b9718d6d2454967bb1ea16b087884b4 P 6a88adec320744be9aea7dc0c0b1bdc0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a88adec320744be9aea7dc0c0b1bdc0, cb97bdf0eac640deacd92387287e0f6a; no voters:
I20260806 04:58:49.972360 25490 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9b9718d6d2454967bb1ea16b087884b4" candidate_uuid: "6a88adec320744be9aea7dc0c0b1bdc0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c01dc087864347b2a8e070a3504b5d14" is_pre_election: true
I20260806 04:58:49.972576 25779 raft_consensus.cc:2781] T 9b9718d6d2454967bb1ea16b087884b4 P 6a88adec320744be9aea7dc0c0b1bdc0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 04:58:49.972736 25779 raft_consensus.cc:482] T 9b9718d6d2454967bb1ea16b087884b4 P 6a88adec320744be9aea7dc0c0b1bdc0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 04:58:49.972877 25779 raft_consensus.cc:3037] T 9b9718d6d2454967bb1ea16b087884b4 P 6a88adec320744be9aea7dc0c0b1bdc0 [term 0 FOLLOWER]: Advancing to term 1
I20260806 04:58:49.972833 25490 raft_consensus.cc:2454] T 9b9718d6d2454967bb1ea16b087884b4 P c01dc087864347b2a8e070a3504b5d14 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a88adec320744be9aea7dc0c0b1bdc0 in term 0.
I20260806 04:58:49.975243 25779 raft_consensus.cc:504] T 9b9718d6d2454967bb1ea16b087884b4 P 6a88adec320744be9aea7dc0c0b1bdc0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a88adec320744be9aea7dc0c0b1bdc0" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 44759 } } peers { permanent_uuid: "cb97bdf0eac640deacd92387287e0f6a" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 45757 } } peers { permanent_uuid: "c01dc087864347b2a8e070a3504b5d14" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 36695 } }
I20260806 04:58:49.975828 25779 leader_election.cc:288] T 9b9718d6d2454967bb1ea16b087884b4 P 6a88adec320744be9aea7dc0c0b1bdc0 [CANDIDATE]: Term 1 election: Requested vote from peers cb97bdf0eac640deacd92387287e0f6a (127.9.96.65:45757), c01dc087864347b2a8e070a3504b5d14 (127.9.96.66:36695)
I20260806 04:58:49.976310 25293 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9b9718d6d2454967bb1ea16b087884b4" candidate_uuid: "6a88adec320744be9aea7dc0c0b1bdc0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb97bdf0eac640deacd92387287e0f6a"
I20260806 04:58:49.976310 25490 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9b9718d6d2454967bb1ea16b087884b4" candidate_uuid: "6a88adec320744be9aea7dc0c0b1bdc0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c01dc087864347b2a8e070a3504b5d14"
I20260806 04:58:49.976504 25293 raft_consensus.cc:3037] T 9b9718d6d2454967bb1ea16b087884b4 P cb97bdf0eac640deacd92387287e0f6a [term 0 FOLLOWER]: Advancing to term 1
I20260806 04:58:49.976504 25490 raft_consensus.cc:3037] T 9b9718d6d2454967bb1ea16b087884b4 P c01dc087864347b2a8e070a3504b5d14 [term 0 FOLLOWER]: Advancing to term 1
I20260806 04:58:49.979741 25293 raft_consensus.cc:2454] T 9b9718d6d2454967bb1ea16b087884b4 P cb97bdf0eac640deacd92387287e0f6a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a88adec320744be9aea7dc0c0b1bdc0 in term 1.
I20260806 04:58:49.979741 25490 raft_consensus.cc:2454] T 9b9718d6d2454967bb1ea16b087884b4 P c01dc087864347b2a8e070a3504b5d14 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a88adec320744be9aea7dc0c0b1bdc0 in term 1.
I20260806 04:58:49.980185 25591 leader_election.cc:302] T 9b9718d6d2454967bb1ea16b087884b4 P 6a88adec320744be9aea7dc0c0b1bdc0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a88adec320744be9aea7dc0c0b1bdc0, cb97bdf0eac640deacd92387287e0f6a; no voters:
I20260806 04:58:49.980502 25779 raft_consensus.cc:2781] T 9b9718d6d2454967bb1ea16b087884b4 P 6a88adec320744be9aea7dc0c0b1bdc0 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 04:58:49.981032 25779 raft_consensus.cc:686] T 9b9718d6d2454967bb1ea16b087884b4 P 6a88adec320744be9aea7dc0c0b1bdc0 [term 1 LEADER]: Becoming Leader. State: Replica: 6a88adec320744be9aea7dc0c0b1bdc0, State: Running, Role: LEADER
I20260806 04:58:49.981536 25779 consensus_queue.cc:237] T 9b9718d6d2454967bb1ea16b087884b4 P 6a88adec320744be9aea7dc0c0b1bdc0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a88adec320744be9aea7dc0c0b1bdc0" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 44759 } } peers { permanent_uuid: "cb97bdf0eac640deacd92387287e0f6a" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 45757 } } peers { permanent_uuid: "c01dc087864347b2a8e070a3504b5d14" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 36695 } }
I20260806 04:58:49.985674 25082 catalog_manager.cc:5495] T 9b9718d6d2454967bb1ea16b087884b4 P 6a88adec320744be9aea7dc0c0b1bdc0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6a88adec320744be9aea7dc0c0b1bdc0 (127.9.96.67). New cstate: current_term: 1 leader_uuid: "6a88adec320744be9aea7dc0c0b1bdc0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a88adec320744be9aea7dc0c0b1bdc0" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 44759 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb97bdf0eac640deacd92387287e0f6a" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 45757 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c01dc087864347b2a8e070a3504b5d14" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 36695 } health_report { overall_health: UNKNOWN } } }
I20260806 04:58:50.011927 25764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:58:50.007191 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60892 (local address 127.9.96.67:44759)
0806 04:58:50.007391 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0806 04:58:50.007395 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 04:58:50.007411 (+ 16us) server_negotiation.cc:407] Connection header received
0806 04:58:50.007593 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:58:50.007597 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:58:50.007671 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:58:50.007797 (+ 126us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 04:58:50.008140 (+ 343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:50.008858 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:50.010091 (+ 1233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:50.010454 (+ 363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:50.010540 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:58:50.010959 (+ 419us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 04:58:50.011497 (+ 538us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 04:58:50.011607 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0806 04:58:50.011681 (+ 74us) server_negotiation.cc:299] Negotiation successful
0806 04:58:50.011739 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260806 04:58:50.019565 25490 raft_consensus.cc:1261] T 9b9718d6d2454967bb1ea16b087884b4 P c01dc087864347b2a8e070a3504b5d14 [term 1 FOLLOWER]: Refusing update from remote peer 6a88adec320744be9aea7dc0c0b1bdc0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 04:58:50.019568 25293 raft_consensus.cc:1261] T 9b9718d6d2454967bb1ea16b087884b4 P cb97bdf0eac640deacd92387287e0f6a [term 1 FOLLOWER]: Refusing update from remote peer 6a88adec320744be9aea7dc0c0b1bdc0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 04:58:50.020668 25785 consensus_queue.cc:1035] T 9b9718d6d2454967bb1ea16b087884b4 P 6a88adec320744be9aea7dc0c0b1bdc0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb97bdf0eac640deacd92387287e0f6a" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 45757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 04:58:50.021152 25779 consensus_queue.cc:1035] T 9b9718d6d2454967bb1ea16b087884b4 P 6a88adec320744be9aea7dc0c0b1bdc0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c01dc087864347b2a8e070a3504b5d14" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 36695 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260806 04:58:50.031262 25361 tablet.cc:2369] T 9b9718d6d2454967bb1ea16b087884b4 P cb97bdf0eac640deacd92387287e0f6a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 04:58:50.048352 25081 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39802:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 04:58:50.048614 25081 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39802:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 04:58:50.052207 25081 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a6642842c39f4f0faa1c228c95d13a1a: Sending DeleteTablet for 3 replicas of tablet 9b9718d6d2454967bb1ea16b087884b4
I20260806 04:58:50.053313 25462 tablet_service.cc:1514] Processing DeleteTablet for tablet 9b9718d6d2454967bb1ea16b087884b4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 04:58:50 UTC) from {username='kudu', principal='kudu/127.9.96.126@KRBTEST.COM'} at 127.0.0.1:41892
I20260806 04:58:50.053300 25267 tablet_service.cc:1514] Processing DeleteTablet for tablet 9b9718d6d2454967bb1ea16b087884b4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 04:58:50 UTC) from {username='kudu', principal='kudu/127.9.96.126@KRBTEST.COM'} at 127.0.0.1:57570
I20260806 04:58:50.054018 25652 tablet_service.cc:1514] Processing DeleteTablet for tablet 9b9718d6d2454967bb1ea16b087884b4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 04:58:50 UTC) from {username='kudu', principal='kudu/127.9.96.126@KRBTEST.COM'} at 127.0.0.1:60882
I20260806 04:58:50.054400 25802 tablet_replica.cc:330] T 9b9718d6d2454967bb1ea16b087884b4 P cb97bdf0eac640deacd92387287e0f6a: stopping tablet replica
I20260806 04:58:50.054885 25802 raft_consensus.cc:2229] T 9b9718d6d2454967bb1ea16b087884b4 P cb97bdf0eac640deacd92387287e0f6a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 04:58:50.055071 25804 tablet_replica.cc:330] T 9b9718d6d2454967bb1ea16b087884b4 P 6a88adec320744be9aea7dc0c0b1bdc0: stopping tablet replica
I20260806 04:58:50.055063 9601 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu pid 25177
I20260806 04:58:50.055440 25804 raft_consensus.cc:2229] T 9b9718d6d2454967bb1ea16b087884b4 P 6a88adec320744be9aea7dc0c0b1bdc0 [term 1 LEADER]: Raft consensus shutting down.
I20260806 04:58:50.055716 25802 raft_consensus.cc:2258] T 9b9718d6d2454967bb1ea16b087884b4 P cb97bdf0eac640deacd92387287e0f6a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 04:58:50.056038 25804 raft_consensus.cc:2258] T 9b9718d6d2454967bb1ea16b087884b4 P 6a88adec320744be9aea7dc0c0b1bdc0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 04:58:50.058080 25804 ts_tablet_manager.cc:1905] T 9b9718d6d2454967bb1ea16b087884b4 P 6a88adec320744be9aea7dc0c0b1bdc0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 04:58:50.058303 25802 ts_tablet_manager.cc:1905] T 9b9718d6d2454967bb1ea16b087884b4 P cb97bdf0eac640deacd92387287e0f6a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 04:58:50.065714 25802 ts_tablet_manager.cc:1918] T 9b9718d6d2454967bb1ea16b087884b4 P cb97bdf0eac640deacd92387287e0f6a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 04:58:50.065951 25802 log.cc:1196] T 9b9718d6d2454967bb1ea16b087884b4 P cb97bdf0eac640deacd92387287e0f6a: Deleting WAL directory at /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785992227050689-9601-0/minicluster-data/ts-0/wal/wals/9b9718d6d2454967bb1ea16b087884b4
I20260806 04:58:50.066546 25802 ts_tablet_manager.cc:1939] T 9b9718d6d2454967bb1ea16b087884b4 P cb97bdf0eac640deacd92387287e0f6a: Deleting consensus metadata
I20260806 04:58:50.067324 25804 ts_tablet_manager.cc:1918] T 9b9718d6d2454967bb1ea16b087884b4 P 6a88adec320744be9aea7dc0c0b1bdc0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 04:58:50.067509 25804 log.cc:1196] T 9b9718d6d2454967bb1ea16b087884b4 P 6a88adec320744be9aea7dc0c0b1bdc0: Deleting WAL directory at /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785992227050689-9601-0/minicluster-data/ts-2/wal/wals/9b9718d6d2454967bb1ea16b087884b4
I20260806 04:58:50.068028 25804 ts_tablet_manager.cc:1939] T 9b9718d6d2454967bb1ea16b087884b4 P 6a88adec320744be9aea7dc0c0b1bdc0: Deleting consensus metadata
I20260806 04:58:50.068723 25069 catalog_manager.cc:4841] TS cb97bdf0eac640deacd92387287e0f6a (127.9.96.65:45757): tablet 9b9718d6d2454967bb1ea16b087884b4 (table test-table [id=35e9e8d83608429c938b740bcc1e5c0a]) successfully deleted
I20260806 04:58:50.069555 25065 catalog_manager.cc:4841] TS 6a88adec320744be9aea7dc0c0b1bdc0 (127.9.96.67:44759): tablet 9b9718d6d2454967bb1ea16b087884b4 (table test-table [id=35e9e8d83608429c938b740bcc1e5c0a]) successfully deleted
I20260806 04:58:50.069798 25803 tablet_replica.cc:330] T 9b9718d6d2454967bb1ea16b087884b4 P c01dc087864347b2a8e070a3504b5d14: stopping tablet replica
I20260806 04:58:50.070089 25803 raft_consensus.cc:2229] T 9b9718d6d2454967bb1ea16b087884b4 P c01dc087864347b2a8e070a3504b5d14 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 04:58:50.070425 25803 raft_consensus.cc:2258] T 9b9718d6d2454967bb1ea16b087884b4 P c01dc087864347b2a8e070a3504b5d14 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 04:58:50.072271 25803 ts_tablet_manager.cc:1905] T 9b9718d6d2454967bb1ea16b087884b4 P c01dc087864347b2a8e070a3504b5d14: Deleting tablet data with delete state TABLET_DATA_DELETED
Aug 06 04:58:50 dist-test-slave-x4qh krb5kdc[25027](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992327, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.96.65@KRBTEST.COM
I20260806 04:58:50.078842 25803 ts_tablet_manager.cc:1918] T 9b9718d6d2454967bb1ea16b087884b4 P c01dc087864347b2a8e070a3504b5d14: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 04:58:50.078995 25803 log.cc:1196] T 9b9718d6d2454967bb1ea16b087884b4 P c01dc087864347b2a8e070a3504b5d14: Deleting WAL directory at /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785992227050689-9601-0/minicluster-data/ts-1/wal/wals/9b9718d6d2454967bb1ea16b087884b4
I20260806 04:58:50.079564 25803 ts_tablet_manager.cc:1939] T 9b9718d6d2454967bb1ea16b087884b4 P c01dc087864347b2a8e070a3504b5d14: Deleting consensus metadata
I20260806 04:58:50.080926 25068 catalog_manager.cc:4841] TS c01dc087864347b2a8e070a3504b5d14 (127.9.96.66:36695): tablet 9b9718d6d2454967bb1ea16b087884b4 (table test-table [id=35e9e8d83608429c938b740bcc1e5c0a]) successfully deleted
I20260806 04:58:50.084474 25766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:58:50.057766 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57578 (local address 127.9.96.65:45757)
0806 04:58:50.057939 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0806 04:58:50.057945 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 04:58:50.066303 (+ 8358us) server_negotiation.cc:407] Connection header received
0806 04:58:50.066740 (+ 437us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:58:50.066746 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:58:50.066819 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:58:50.066895 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:58:50.069359 (+ 2464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:50.070462 (+ 1103us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:50.072266 (+ 1804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:50.072913 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:50.073051 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:58:50.078030 (+ 4979us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:58:50.078043 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:58:50.078060 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:58:50.078097 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:58:50.081685 (+ 3588us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:58:50.082328 (+ 643us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:58:50.082335 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:58:50.082339 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:58:50.082479 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:58:50.082874 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:58:50.082880 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:58:50.082884 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:58:50.083312 (+ 428us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:58:50.083687 (+ 375us) server_negotiation.cc:1035] Waiting for connection context
0806 04:58:50.083863 (+ 176us) server_negotiation.cc:299] Negotiation successful
0806 04:58:50.084088 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260806 04:58:50.085167 25354 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 04:58:50.172621 9601 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu with pid 25177
I20260806 04:58:50.196729 9601 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu pid 25368
Aug 06 04:58:50 dist-test-slave-x4qh krb5kdc[25027](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992327, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.96.66@KRBTEST.COM
I20260806 04:58:50.212322 25767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:58:50.197524 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41908 (local address 127.9.96.66:36695)
0806 04:58:50.197712 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0806 04:58:50.197718 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 04:58:50.197793 (+ 75us) server_negotiation.cc:407] Connection header received
0806 04:58:50.198015 (+ 222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:58:50.198027 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:58:50.198099 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:58:50.198209 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:58:50.199521 (+ 1312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:50.200273 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:50.201512 (+ 1239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:50.201917 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:50.202068 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:58:50.206394 (+ 4326us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:58:50.206408 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:58:50.206423 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:58:50.206457 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:58:50.209447 (+ 2990us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:58:50.210146 (+ 699us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:58:50.210153 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:58:50.210155 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:58:50.210291 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:58:50.210664 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:58:50.210670 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:58:50.210673 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:58:50.211050 (+ 377us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:58:50.211414 (+ 364us) server_negotiation.cc:1035] Waiting for connection context
0806 04:58:50.211749 (+ 335us) server_negotiation.cc:299] Negotiation successful
0806 04:58:50.211979 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260806 04:58:50.212983 25546 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 04:58:50.302387 9601 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu with pid 25368
I20260806 04:58:50.326591 9601 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu pid 25567
Aug 06 04:58:50 dist-test-slave-x4qh krb5kdc[25027](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992327, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.96.67@KRBTEST.COM
I20260806 04:58:50.343114 25764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:58:50.327430 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38052 (local address 127.9.96.67:44759)
0806 04:58:50.327591 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0806 04:58:50.327597 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 04:58:50.327668 (+ 71us) server_negotiation.cc:407] Connection header received
0806 04:58:50.327905 (+ 237us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:58:50.327911 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:58:50.327989 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:58:50.328129 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:58:50.329422 (+ 1293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:50.330264 (+ 842us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:50.331661 (+ 1397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:58:50.332211 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:58:50.332347 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:58:50.336763 (+ 4416us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:58:50.336775 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:58:50.336789 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:58:50.336830 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:58:50.340223 (+ 3393us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:58:50.340944 (+ 721us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:58:50.340950 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:58:50.340953 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:58:50.341104 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:58:50.341496 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:58:50.341500 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:58:50.341502 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:58:50.341888 (+ 386us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:58:50.342274 (+ 386us) server_negotiation.cc:1035] Waiting for connection context
0806 04:58:50.342502 (+ 228us) server_negotiation.cc:299] Negotiation successful
0806 04:58:50.342727 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260806 04:58:50.343845 25740 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 04:58:50.465086 9601 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu with pid 25567
I20260806 04:58:50.490283 9601 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu pid 25045
I20260806 04:58:50.491484 25128 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 04:58:50.599965 9601 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu with pid 25045
2026-08-06T04:58:50Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (4388 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785992227050689-9601-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 04:58:50 dist-test-slave-x4qh krb5kdc[25816](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 04:58:50 dist-test-slave-x4qh krb5kdc[25816](info): set up 2 sockets
Aug 06 04:58:50 dist-test-slave-x4qh krb5kdc[25816](info): commencing operation
krb5kdc: starting...
W20260806 04:58:51.954291 9601 mini_kdc.cc:121] Time spent starting KDC: real 1.302s user 0.001s sys 0.033s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785992227050689-9601-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785992227050689-9601-0/krb5kdc/test-user.keytab.
Aug 06 04:58:52 dist-test-slave-x4qh krb5kdc[25816](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992332, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T04:58:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T04:58:52Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.9.96.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.9.96.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.9.96.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785992227050689-9601-0/krb5kdc/rangeradmin_127.9.96.84@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.9.96.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785992227050689-9601-0/krb5kdc/rangeradmin_127.9.96.84@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.9.96.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.9.96.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.9.96.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785992227050689-9601-0/krb5kdc/rangerlookup_127.9.96.84@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.9.96.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785992227050689-9601-0/krb5kdc/rangerlookup_127.9.96.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.9.96.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.9.96.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.9.96.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785992227050689-9601-0/krb5kdc/HTTP_127.9.96.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.9.96.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785992227050689-9601-0/krb5kdc/HTTP_127.9.96.84@KRBTEST.COM.keytab.
I20260806 04:58:52.300277 9601 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785992227050689-9601-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskOfb9uZ/build/asan/bin/postgres/pg_ctl -D /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785992227050689-9601-0/minicluster-data/postgres -l logfile start
2026-08-06 04:58:54.737 UTC [25858] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-06 04:58:54.737 UTC [25858] LOG: listening on IPv4 address "127.9.96.84", port 48811
2026-08-06 04:58:54.741 UTC [25858] LOG: listening on Unix socket "/tmp/.s.PGSQL.48811"
2026-08-06 04:58:54.764 UTC [25861] LOG: database system was shut down at 2026-08-06 04:58:53 UTC
2026-08-06 04:58:54.767 UTC [25858] LOG: database system is ready to accept connections
I20260806 04:58:55.926967 9601 mini_postgres.cc:96] Postgres bound to 48811
2026-08-06 04:58:55.951 UTC [25877] FATAL: database "slave" does not exist
127.9.96.84:48811 - accepting connections
I20260806 04:58:55.952368 9601 mini_ranger.cc:161] Starting Ranger...
I20260806 04:58:55.984038 9601 mini_ranger.cc:84] Created miniranger Postgres user
I20260806 04:58:56.439452 9601 mini_ranger.cc:87] Created ranger Postgres database
I20260806 04:58:56.439632 9601 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785992227050689-9601-0/minicluster-data/ranger-admin
2026-08-06 04:58:56,767 [I] DB FLAVOR :POSTGRES
2026-08-06 04:58:56,768 [I] --------- Verifying Ranger DB connection ---------
2026-08-06 04:58:56,768 [I] Checking connection..
2026-08-06 04:58:56,768 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskOfb9uZ/build/asan/bin/postgresql.jar:/tmp/dist-test-taskOfb9uZ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.96.84:48811/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-06 04:58:57,039 [I] Checking connection passed.
2026-08-06 04:58:57,039 [I] --------- Verifying version history table ---------
2026-08-06 04:58:57,039 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskOfb9uZ/build/asan/bin/postgresql.jar:/tmp/dist-test-taskOfb9uZ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.96.84:48811/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-06 04:58:57,307 [I] Table x_db_version_h does not exist in database ranger
2026-08-06 04:58:57,307 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskOfb9uZ/build/asan/bin/postgresql.jar:/tmp/dist-test-taskOfb9uZ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.96.84:48811/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-06 04:58:57,557 [I] Table x_db_version_h does not exist in database ranger
2026-08-06 04:58:57,558 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-06 04:58:57,558 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskOfb9uZ/build/asan/bin/postgresql.jar:/tmp/dist-test-taskOfb9uZ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.96.84:48811/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskOfb9uZ/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-06 04:58:57.809 UTC [25976] WARNING: there is no transaction in progress
2026-08-06 04:58:57,817 [I] create_dbversion_catalog.sql file imported successfully
2026-08-06 04:58:57,818 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskOfb9uZ/build/asan/bin/postgresql.jar:/tmp/dist-test-taskOfb9uZ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.96.84:48811/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-06 04:58:58,086 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-06 04:58:58,086 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-06 04:58:58,086 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskOfb9uZ/build/asan/bin/postgresql.jar:/tmp/dist-test-taskOfb9uZ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.96.84:48811/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-06 04:58:58,350 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskOfb9uZ/build/asan/bin/postgresql.jar:/tmp/dist-test-taskOfb9uZ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.96.84:48811/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-06 04:58:58,608 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskOfb9uZ/build/asan/bin/postgresql.jar:/tmp/dist-test-taskOfb9uZ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.96.84:48811/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-x4qh.c.gcp-upstream.internal','N') ;"
2026-08-06 04:58:58,850 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskOfb9uZ/build/asan/bin/postgresql.jar:/tmp/dist-test-taskOfb9uZ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.96.84:48811/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-06 04:58:59,093 [I] Table x_portal_user does not exist in database ranger
2026-08-06 04:58:59,093 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskOfb9uZ/build/asan/bin/postgresql.jar:/tmp/dist-test-taskOfb9uZ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.96.84:48811/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-06 04:58:59,335 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-06 04:58:59,335 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskOfb9uZ/build/asan/bin/postgresql.jar:/tmp/dist-test-taskOfb9uZ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.96.84:48811/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-06 04:58:59,587 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskOfb9uZ/build/asan/bin/postgresql.jar:/tmp/dist-test-taskOfb9uZ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.96.84:48811/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-06 04:58:59,826 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-06 04:58:59,827 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskOfb9uZ/build/asan/bin/postgresql.jar:/tmp/dist-test-taskOfb9uZ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.96.84:48811/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskOfb9uZ/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-06 04:59:00.704 UTC [26136] WARNING: there is no transaction in progress
2026-08-06 04:59:00.719 UTC [26136] WARNING: there is no transaction in progress
2026-08-06 04:59:00.733 UTC [26136] WARNING: there is no transaction in progress
2026-08-06 04:59:00.748 UTC [26136] WARNING: there is no transaction in progress
2026-08-06 04:59:00.765 UTC [26136] WARNING: there is no transaction in progress
2026-08-06 04:59:00.785 UTC [26136] WARNING: there is no transaction in progress
2026-08-06 04:59:00.847 UTC [26136] WARNING: there is no transaction in progress
2026-08-06 04:59:00.854 UTC [26136] WARNING: there is no transaction in progress
2026-08-06 04:59:00.867 UTC [26136] WARNING: there is no transaction in progress
2026-08-06 04:59:00.877 UTC [26136] WARNING: there is no transaction in progress
2026-08-06 04:59:00.886 UTC [26136] WARNING: there is no transaction in progress
2026-08-06 04:59:00.898 UTC [26136] WARNING: there is no transaction in progress
2026-08-06 04:59:00.907 UTC [26136] WARNING: there is no transaction in progress
2026-08-06 04:59:00.913 UTC [26136] WARNING: there is no transaction in progress
2026-08-06 04:59:00.919 UTC [26136] WARNING: there is no transaction in progress
2026-08-06 04:59:00.925 UTC [26136] WARNING: there is no transaction in progress
2026-08-06 04:59:01,519 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-06 04:59:01,519 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskOfb9uZ/build/asan/bin/postgresql.jar:/tmp/dist-test-taskOfb9uZ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.96.84:48811/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-06 04:59:01,770 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-06 04:59:01,771 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskOfb9uZ/build/asan/bin/postgresql.jar:/tmp/dist-test-taskOfb9uZ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.96.84:48811/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-06 04:59:02,035 [I] Table x_portal_user already exists in database 'ranger'
2026-08-06 04:59:02,035 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskOfb9uZ/build/asan/bin/postgresql.jar:/tmp/dist-test-taskOfb9uZ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.96.84:48811/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-06 04:59:02,296 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-06 04:59:02,297 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskOfb9uZ/build/asan/bin/postgresql.jar:/tmp/dist-test-taskOfb9uZ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.96.84:48811/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-06 04:59:02,547 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskOfb9uZ/build/asan/bin/postgresql.jar:/tmp/dist-test-taskOfb9uZ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.96.84:48811/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-06 04:59:02,785 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskOfb9uZ/build/asan/bin/postgresql.jar:/tmp/dist-test-taskOfb9uZ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.96.84:48811/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-x4qh.c.gcp-upstream.internal';"
2026-08-06 04:59:03,019 [I] CORE_DB_SCHEMA import status has been updated
2026-08-06 04:59:03,019 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskOfb9uZ/build/asan/bin/postgresql.jar:/tmp/dist-test-taskOfb9uZ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.96.84:48811/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-06 04:59:03,273 [I] DB_PATCHES have already been applied
I20260806 04:59:03.279769 9601 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785992227050689-9601-0/minicluster-data/ranger-admin:/tmp/dist-test-taskOfb9uZ/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskOfb9uZ/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskOfb9uZ/build/asan/bin/postgresql.jar:/tmp/dist-test-taskOfb9uZ/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260806 04:59:03.279877 9601 mini_ranger.cc:193] Using host: 127.9.96.84
I20260806 04:59:03.300225 9601 mini_ranger.cc:227] Ranger admin URL: http://127.9.96.84:48185
Aug 06, 2026 4:59:03 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 06, 2026 4:59:03 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 06, 2026 4:59:03 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785992227050689-9601-0/minicluster-data/ranger-admin/ews/webapp
Aug 06, 2026 4:59:03 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785992227050689-9601-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 06, 2026 4:59:03 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785992227050689-9601-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 06, 2026 4:59:04 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785992227050689-9601-0/minicluster-data/ranger-admin/ews/webapp].
Aug 06, 2026 4:59:04 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260806 04:59:04.692267 9601 mini_ranger.cc:160] Time spent starting Ranger: real 8.740s user 0.003s sys 0.055s
Aug 06, 2026 4:59:25 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 06, 2026 4:59:25 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.MetricsREST
Aug 06, 2026 4:59:25 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 06, 2026 4:59:25 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 06, 2026 4:59:25 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 06, 2026 4:59:25 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 06, 2026 4:59:25 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 06, 2026 4:59:25 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 06, 2026 4:59:25 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 06, 2026 4:59:25 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 06, 2026 4:59:25 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 06, 2026 4:59:25 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 06, 2026 4:59:25 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 06, 2026 4:59:25 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 06, 2026 4:59:25 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 06, 2026 4:59:25 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 06, 2026 4:59:25 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 06, 2026 4:59:26 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260806 04:59:26.671015 9601 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.9.96.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.9.96.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.9.96.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785992227050689-9601-0/krb5kdc/rangerkms_127.9.96.84@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.9.96.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785992227050689-9601-0/krb5kdc/rangerkms_127.9.96.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.9.96.84@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.9.96.84@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.9.96.84@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785992227050689-9601-0/krb5kdc/HTTP_127.9.96.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.9.96.84@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785992227050689-9601-0/krb5kdc/HTTP_127.9.96.84@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 06 04:59:26 dist-test-slave-x4qh krb5kdc[25816](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992366, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260806 04:59:26.875173 9601 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260806 04:59:26.897557 9601 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260806 04:59:27.662025 9601 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260806 04:59:27.662174 9601 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785992227050689-9601-0/minicluster-data/ranger-kms
I20260806 04:59:27.662330 9601 mini_ranger_kms.cc:227] Using postgres at 127.9.96.84:48811
2026-08-06 04:59:28,146 [I] DB FLAVOR :POSTGRES
2026-08-06 04:59:28,146 [I] --------- Verifying Ranger DB connection ---------
2026-08-06 04:59:28,147 [I] Checking connection
2026-08-06 04:59:28,147 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskOfb9uZ/build/asan/bin/postgresql.jar:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785992227050689-9601-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.96.84:48811/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-06 04:59:28,405 [I] connection success
2026-08-06 04:59:28,405 [I] --------- Verifying Ranger DB tables ---------
2026-08-06 04:59:28,406 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-06 04:59:28,406 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskOfb9uZ/build/asan/bin/postgresql.jar:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785992227050689-9601-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.96.84:48811/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-06 04:59:28,641 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-06 04:59:28,642 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-06 04:59:28,642 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-06 04:59:28,642 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskOfb9uZ/build/asan/bin/postgresql.jar:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785992227050689-9601-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.96.84:48811/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785992227050689-9601-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-06 04:59:28,900 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260806 04:59:29.390551 9601 mini_ranger_kms.cc:310] Created kms service
I20260806 04:59:29.580255 9601 mini_ranger_kms.cc:326] Created kudu user
I20260806 04:59:29.625002 9601 mini_ranger_kms.cc:343] Created rangerkms user
Aug 06 04:59:29 dist-test-slave-x4qh krb5kdc[25816](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992366, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.9.96.84@KRBTEST.COM
I20260806 04:59:29.996316 9601 mini_ranger_kms.cc:384] Added ranger policy
I20260806 04:59:29.996701 9601 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785992227050689-9601-0/minicluster-data/ranger-kms:/tmp/dist-test-taskOfb9uZ/build/asan/bin/postgresql.jar:/tmp/dist-test-taskOfb9uZ/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskOfb9uZ/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260806 04:59:29.996760 9601 mini_ranger_kms.cc:240] Using host: 127.9.96.84
I20260806 04:59:30.021225 9601 mini_ranger_kms.cc:276] Ranger KMS PID: 26407
I20260806 04:59:30.021430 9601 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.9.96.84:39207
Aug 06, 2026 4:59:31 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 06, 2026 4:59:31 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 06, 2026 4:59:31 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785992227050689-9601-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 06, 2026 4:59:31 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785992227050689-9601-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 06, 2026 4:59:31 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785992227050689-9601-0/minicluster-data/ranger-kms/ews/webapp].
Aug 06, 2026 4:59:31 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 06 04:59:32 dist-test-slave-x4qh krb5kdc[25816](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1785992372, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.9.96.84@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 04:59:32.759795 9601 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.885s user 0.143s sys 0.375s
I20260806 04:59:32.760183 9601 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260806 04:59:32.760422 9601 mini_ranger_kms.cc:401] 127.9.96.84:39207/kms/v1/keys
Aug 06 04:59:32 dist-test-slave-x4qh krb5kdc[25816](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992372, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.9.96.84@KRBTEST.COM for HTTP/127.9.96.84@KRBTEST.COM
Aug 06 04:59:33 dist-test-slave-x4qh krb5kdc[25816](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992372, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.9.96.84@KRBTEST.COM for HTTP/127.9.96.84@KRBTEST.COM
Aug 06 04:59:35 dist-test-slave-x4qh krb5kdc[25816](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992375, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.9.96.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.96.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.96.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.126.keytab.
Entry for principal kudu/127.9.96.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.126.keytab.
I20260806 04:59:35.543777 9601 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
/tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785992227050689-9601-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785992227050689-9601-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785992227050689-9601-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785992227050689-9601-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.96.126:41885
--webserver_interface=127.9.96.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.126.keytab
--principal=kudu/127.9.96.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.96.84:43827
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.96.126:41885
--ranger_config_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785992227050689-9601-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.9.96.84:39207/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785992227050689-9601-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785992227050689-9601-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785992227050689-9601-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 04:59:35.889856 26483 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 04:59:35.890163 26483 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 04:59:35.890345 26483 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 04:59:35.897285 26483 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 04:59:35.897419 26483 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 04:59:35.897516 26483 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 04:59:35.897611 26483 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 04:59:35.897696 26483 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 04:59:35.907496 26483 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.96.84:43827
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785992227050689-9601-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785992227050689-9601-0/minicluster-data/master-0/wal
--ranger_kms_url=127.9.96.84:39207/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.9.96.126:41885
--ranger_config_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785992227050689-9601-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.126.keytab
--principal=kudu/127.9.96.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.96.126:41885
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785992227050689-9601-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.96.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785992227050689-9601-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 03:56:12 UTC on ccycloud.quasar-guijbm.root.comops.site
build id 4569618
ASAN enabled
I20260806 04:59:35.908712 26483 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 04:59:35.909991 26483 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 04:59:35.922329 26492 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:59:35.922343 26490 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:59:35.922448 26489 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:59:35.923044 26483 server_base.cc:1034] running on GCE node
I20260806 04:59:35.923837 26483 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 04:59:35.925141 26483 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 04:59:35.926363 26483 hybrid_clock.cc:648] HybridClock initialized: now 1785992375926327 us; error 44 us; skew 500 ppm
I20260806 04:59:35.926692 26483 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 04:59:35 dist-test-slave-x4qh krb5kdc[25816](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992375, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.96.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 04:59:35.934600 26483 init.cc:377] Logged in from keytab as kudu/127.9.96.126@KRBTEST.COM (short username kudu)
I20260806 04:59:35.936106 26483 webserver.cc:460] Webserver started at http://127.9.96.126:40225/ using document root <none> and password file <none>
I20260806 04:59:35.936674 26483 fs_manager.cc:359] Metadata directory not provided
I20260806 04:59:35.936767 26483 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 04:59:35.937069 26483 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 04:59:35 dist-test-slave-x4qh krb5kdc[25816](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992375, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.96.126@KRBTEST.COM for HTTP/127.9.96.84@KRBTEST.COM
I20260806 04:59:35.988981 26483 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785992227050689-9601-0/minicluster-data/master-0/data/instance:
uuid: "ad7852ba3f9c4377baf400ad6f80588f"
format_stamp: "Formatted at 2026-08-06 04:59:35 on dist-test-slave-x4qh"
server_key: "618d146e14d6cce464b52db93f4e8bb9"
server_key_iv: "c408ed783a207a7b8a181e69d6b209fd"
server_key_version: "kuduclusterkey@0"
I20260806 04:59:35.989773 26483 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785992227050689-9601-0/minicluster-data/master-0/wal/instance:
uuid: "ad7852ba3f9c4377baf400ad6f80588f"
format_stamp: "Formatted at 2026-08-06 04:59:35 on dist-test-slave-x4qh"
server_key: "618d146e14d6cce464b52db93f4e8bb9"
server_key_iv: "c408ed783a207a7b8a181e69d6b209fd"
server_key_version: "kuduclusterkey@0"
I20260806 04:59:35.995234 26483 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260806 04:59:36.011229 26503 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 04:59:36.012790 26483 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260806 04:59:36.012991 26483 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785992227050689-9601-0/minicluster-data/master-0/data,/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785992227050689-9601-0/minicluster-data/master-0/wal
uuid: "ad7852ba3f9c4377baf400ad6f80588f"
format_stamp: "Formatted at 2026-08-06 04:59:35 on dist-test-slave-x4qh"
server_key: "618d146e14d6cce464b52db93f4e8bb9"
server_key_iv: "c408ed783a207a7b8a181e69d6b209fd"
server_key_version: "kuduclusterkey@0"
I20260806 04:59:36.013216 26483 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785992227050689-9601-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785992227050689-9601-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785992227050689-9601-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 04:59:36.038686 26483 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 04:59:36.042613 26483 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 04:59:36.042946 26483 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 04:59:36.061990 26483 rpc_server.cc:307] RPC server started. Bound to: 127.9.96.126:41885
I20260806 04:59:36.062008 26579 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.96.126:41885 every 8 connection(s)
I20260806 04:59:36.063293 26483 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785992227050689-9601-0/minicluster-data/master-0/data/info.pb
I20260806 04:59:36.065886 9601 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu as pid 26483
I20260806 04:59:36.066087 9601 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785992227050689-9601-0/minicluster-data/master-0/wal/instance
I20260806 04:59:36.066720 9601 external_mini_cluster.cc:1428] Setting key 4ba73e443efce6ce4e9f07931564a193
I20260806 04:59:36.067359 26582 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 04:59:36.080968 26582 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ad7852ba3f9c4377baf400ad6f80588f: Bootstrap starting.
Aug 06 04:59:36 dist-test-slave-x4qh krb5kdc[25816](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992375, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.96.126@KRBTEST.COM
I20260806 04:59:36.084820 26582 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ad7852ba3f9c4377baf400ad6f80588f: Neither blocks nor log segments found. Creating new log.
I20260806 04:59:36.086185 26582 log.cc:824] T 00000000000000000000000000000000 P ad7852ba3f9c4377baf400ad6f80588f: Log is configured to *not* fsync() on all Append() calls
I20260806 04:59:36.089578 26582 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ad7852ba3f9c4377baf400ad6f80588f: No bootstrap required, opened a new log
I20260806 04:59:36.090510 26586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:59:36.068677 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58490 (local address 127.9.96.126:41885)
0806 04:59:36.069504 (+ 827us) server_negotiation.cc:206] Beginning negotiation
0806 04:59:36.069519 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 04:59:36.069566 (+ 47us) server_negotiation.cc:407] Connection header received
0806 04:59:36.072139 (+ 2573us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:59:36.072166 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:59:36.072593 (+ 427us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:59:36.073005 (+ 412us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:59:36.074085 (+ 1080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:59:36.076080 (+ 1995us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:59:36.077211 (+ 1131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:59:36.078284 (+ 1073us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:59:36.078483 (+ 199us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:59:36.082094 (+ 3611us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:59:36.082104 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:59:36.082121 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:59:36.082166 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:59:36.086034 (+ 3868us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:59:36.086732 (+ 698us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:59:36.086744 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:59:36.086754 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:59:36.086965 (+ 211us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:59:36.087471 (+ 506us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:59:36.087475 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:59:36.087477 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:59:36.088137 (+ 660us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:59:36.088406 (+ 269us) server_negotiation.cc:1035] Waiting for connection context
0806 04:59:36.088788 (+ 382us) server_negotiation.cc:299] Negotiation successful
0806 04:59:36.089171 (+ 383us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":597,"thread_start_us":354,"threads_started":1}
I20260806 04:59:36.097443 26582 raft_consensus.cc:348] T 00000000000000000000000000000000 P ad7852ba3f9c4377baf400ad6f80588f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad7852ba3f9c4377baf400ad6f80588f" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 41885 } }
I20260806 04:59:36.097754 26582 raft_consensus.cc:374] T 00000000000000000000000000000000 P ad7852ba3f9c4377baf400ad6f80588f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 04:59:36.097826 26582 raft_consensus.cc:729] T 00000000000000000000000000000000 P ad7852ba3f9c4377baf400ad6f80588f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ad7852ba3f9c4377baf400ad6f80588f, State: Initialized, Role: FOLLOWER
I20260806 04:59:36.098349 26582 consensus_queue.cc:260] T 00000000000000000000000000000000 P ad7852ba3f9c4377baf400ad6f80588f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad7852ba3f9c4377baf400ad6f80588f" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 41885 } }
I20260806 04:59:36.098541 26582 raft_consensus.cc:388] T 00000000000000000000000000000000 P ad7852ba3f9c4377baf400ad6f80588f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 04:59:36.098630 26582 raft_consensus.cc:482] T 00000000000000000000000000000000 P ad7852ba3f9c4377baf400ad6f80588f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 04:59:36.098753 26582 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ad7852ba3f9c4377baf400ad6f80588f [term 0 FOLLOWER]: Advancing to term 1
I20260806 04:59:36.100831 26582 raft_consensus.cc:504] T 00000000000000000000000000000000 P ad7852ba3f9c4377baf400ad6f80588f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad7852ba3f9c4377baf400ad6f80588f" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 41885 } }
I20260806 04:59:36.101265 26582 leader_election.cc:302] T 00000000000000000000000000000000 P ad7852ba3f9c4377baf400ad6f80588f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ad7852ba3f9c4377baf400ad6f80588f; no voters:
I20260806 04:59:36.101745 26582 leader_election.cc:288] T 00000000000000000000000000000000 P ad7852ba3f9c4377baf400ad6f80588f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 04:59:36.101922 26588 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ad7852ba3f9c4377baf400ad6f80588f [term 1 FOLLOWER]: Leader election won for term 1
I20260806 04:59:36.102893 26588 raft_consensus.cc:686] T 00000000000000000000000000000000 P ad7852ba3f9c4377baf400ad6f80588f [term 1 LEADER]: Becoming Leader. State: Replica: ad7852ba3f9c4377baf400ad6f80588f, State: Running, Role: LEADER
I20260806 04:59:36.103410 26588 consensus_queue.cc:237] T 00000000000000000000000000000000 P ad7852ba3f9c4377baf400ad6f80588f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad7852ba3f9c4377baf400ad6f80588f" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 41885 } }
I20260806 04:59:36.103953 26582 sys_catalog.cc:564] T 00000000000000000000000000000000 P ad7852ba3f9c4377baf400ad6f80588f [sys.catalog]: configured and running, proceeding with master startup.
I20260806 04:59:36.106197 26589 sys_catalog.cc:455] T 00000000000000000000000000000000 P ad7852ba3f9c4377baf400ad6f80588f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ad7852ba3f9c4377baf400ad6f80588f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad7852ba3f9c4377baf400ad6f80588f" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 41885 } } }
I20260806 04:59:36.106555 26589 sys_catalog.cc:458] T 00000000000000000000000000000000 P ad7852ba3f9c4377baf400ad6f80588f [sys.catalog]: This master's current role is: LEADER
I20260806 04:59:36.107048 26591 sys_catalog.cc:455] T 00000000000000000000000000000000 P ad7852ba3f9c4377baf400ad6f80588f [sys.catalog]: SysCatalogTable state changed. Reason: New leader ad7852ba3f9c4377baf400ad6f80588f. Latest consensus state: current_term: 1 leader_uuid: "ad7852ba3f9c4377baf400ad6f80588f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad7852ba3f9c4377baf400ad6f80588f" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 41885 } } }
I20260806 04:59:36.107359 26591 sys_catalog.cc:458] T 00000000000000000000000000000000 P ad7852ba3f9c4377baf400ad6f80588f [sys.catalog]: This master's current role is: LEADER
I20260806 04:59:36.108932 26602 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 04:59:36.113848 26602 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 04:59:36.114596 26582 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785992227050689-9601-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260806 04:59:36.132944 26602 catalog_manager.cc:1347] Generated new cluster ID: 59b7bb1b475a471c950855b1a9dab47a
I20260806 04:59:36.133100 26602 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 04:59:36.147482 26602 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 04:59:36.149406 26602 catalog_manager.cc:1504] Loading token signing keys...
I20260806 04:59:36.164947 26602 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ad7852ba3f9c4377baf400ad6f80588f: Generated new TSK 0
I20260806 04:59:36.165948 26602 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 04:59:36.837194 26647 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
WARNING: no policy specified for kudu/127.9.96.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.96.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.96.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.65.keytab.
Entry for principal kudu/127.9.96.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.65.keytab.
I20260806 04:59:36.910542 9601 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
/tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785992227050689-9601-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785992227050689-9601-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785992227050689-9601-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785992227050689-9601-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.9.96.84:39207/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.96.65:0
--local_ip_for_outbound_sockets=127.9.96.65
--webserver_interface=127.9.96.65
--webserver_port=0
--tserver_master_addrs=127.9.96.126:41885
--keytab_file=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.65.keytab
--principal=kudu/127.9.96.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.96.84:43827
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785992227050689-9601-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785992227050689-9601-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785992227050689-9601-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 04:59:37 dist-test-slave-x4qh krb5kdc[25816](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1785992377, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.96.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260806 04:59:37.368352 26650 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 04:59:37.368683 26650 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 04:59:37.368875 26650 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 04:59:37.379701 26650 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 04:59:37.379897 26650 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 04:59:37.380095 26650 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.96.65
I20260806 04:59:37.395689 26650 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.96.84:43827
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785992227050689-9601-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785992227050689-9601-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.9.96.84:39207/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.65.keytab
--principal=kudu/127.9.96.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.96.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785992227050689-9601-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.96.65
--webserver_port=0
--tserver_master_addrs=127.9.96.126:41885
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.96.65
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785992227050689-9601-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 03:56:12 UTC on ccycloud.quasar-guijbm.root.comops.site
build id 4569618
ASAN enabled
I20260806 04:59:37.397578 26650 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 04:59:37.399433 26650 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 04:59:37.422333 26673 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:59:37.423198 26674 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:59:37.441010 26678 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:59:37.441596 26650 server_base.cc:1034] running on GCE node
I20260806 04:59:37.442404 26650 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 04:59:37.443722 26650 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 04:59:37.445003 26650 hybrid_clock.cc:648] HybridClock initialized: now 1785992377444937 us; error 50 us; skew 500 ppm
I20260806 04:59:37.445418 26650 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 04:59:37 dist-test-slave-x4qh krb5kdc[25816](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992377, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.96.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 04:59:37.449887 26650 init.cc:377] Logged in from keytab as kudu/127.9.96.65@KRBTEST.COM (short username kudu)
I20260806 04:59:37.451891 26650 webserver.cc:460] Webserver started at http://127.9.96.65:38461/ using document root <none> and password file <none>
I20260806 04:59:37.452838 26650 fs_manager.cc:359] Metadata directory not provided
I20260806 04:59:37.452989 26650 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 04:59:37.453322 26650 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 04:59:37 dist-test-slave-x4qh krb5kdc[25816](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992377, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.96.65@KRBTEST.COM for HTTP/127.9.96.84@KRBTEST.COM
I20260806 04:59:37.485008 26650 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785992227050689-9601-0/minicluster-data/ts-0/data/instance:
uuid: "f3d52b1be46b435ca76dfde46d3a93b2"
format_stamp: "Formatted at 2026-08-06 04:59:37 on dist-test-slave-x4qh"
server_key: "620ba762e0a15e8faefe4eb9c58f0d8e"
server_key_iv: "a39f95c096651e4d5616998c8e9fb6f2"
server_key_version: "kuduclusterkey@0"
I20260806 04:59:37.486258 26650 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785992227050689-9601-0/minicluster-data/ts-0/wal/instance:
uuid: "f3d52b1be46b435ca76dfde46d3a93b2"
format_stamp: "Formatted at 2026-08-06 04:59:37 on dist-test-slave-x4qh"
server_key: "620ba762e0a15e8faefe4eb9c58f0d8e"
server_key_iv: "a39f95c096651e4d5616998c8e9fb6f2"
server_key_version: "kuduclusterkey@0"
I20260806 04:59:37.500564 26650 fs_manager.cc:691] Time spent creating directory manager: real 0.014s user 0.007s sys 0.004s
I20260806 04:59:37.523612 26689 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 04:59:37.525642 26650 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.003s
I20260806 04:59:37.525936 26650 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785992227050689-9601-0/minicluster-data/ts-0/data,/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785992227050689-9601-0/minicluster-data/ts-0/wal
uuid: "f3d52b1be46b435ca76dfde46d3a93b2"
format_stamp: "Formatted at 2026-08-06 04:59:37 on dist-test-slave-x4qh"
server_key: "620ba762e0a15e8faefe4eb9c58f0d8e"
server_key_iv: "a39f95c096651e4d5616998c8e9fb6f2"
server_key_version: "kuduclusterkey@0"
I20260806 04:59:37.526316 26650 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785992227050689-9601-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785992227050689-9601-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785992227050689-9601-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 04:59:37.557804 26650 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 04:59:37.563063 26650 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 04:59:37.563465 26650 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 04:59:37.579840 26650 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 04:59:37.586722 26650 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 04:59:37.586827 26650 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 04:59:37.587145 26650 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 04:59:37.587277 26650 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 04:59:37 dist-test-slave-x4qh krb5kdc[25816](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992377, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.96.126@KRBTEST.COM for HTTP/127.9.96.84@KRBTEST.COM
I20260806 04:59:37.890147 26650 rpc_server.cc:307] RPC server started. Bound to: 127.9.96.65:38267
I20260806 04:59:37.890203 26869 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.96.65:38267 every 8 connection(s)
I20260806 04:59:37.891932 26650 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785992227050689-9601-0/minicluster-data/ts-0/data/info.pb
I20260806 04:59:37.893290 9601 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu as pid 26650
I20260806 04:59:37.893500 9601 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785992227050689-9601-0/minicluster-data/ts-0/wal/instance
I20260806 04:59:37.894384 9601 external_mini_cluster.cc:1428] Setting key 48218d48ca8b74a584d46493efa527a4
Aug 06 04:59:37 dist-test-slave-x4qh krb5kdc[25816](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992377, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.96.65@KRBTEST.COM for kudu/127.9.96.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.9.96.66@KRBTEST.COM; defaulting to no policy
I20260806 04:59:37.931936 26874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:59:37.898988 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.96.65:35167 (local address 127.9.96.126:41885)
0806 04:59:37.909532 (+ 10544us) server_negotiation.cc:206] Beginning negotiation
0806 04:59:37.909540 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 04:59:37.910705 (+ 1165us) server_negotiation.cc:407] Connection header received
0806 04:59:37.913519 (+ 2814us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:59:37.913525 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:59:37.913616 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:59:37.913698 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:59:37.915804 (+ 2106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:59:37.916983 (+ 1179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:59:37.919343 (+ 2360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:59:37.920015 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:59:37.920107 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:59:37.924565 (+ 4458us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:59:37.924572 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:59:37.924576 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:59:37.924614 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:59:37.928127 (+ 3513us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:59:37.929050 (+ 923us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:59:37.929056 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:59:37.929059 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:59:37.929221 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:59:37.929752 (+ 531us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:59:37.929757 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:59:37.929761 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:59:37.930116 (+ 355us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:59:37.930269 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0806 04:59:37.931220 (+ 951us) server_negotiation.cc:299] Negotiation successful
0806 04:59:37.931539 (+ 319us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10391,"thread_start_us":10237,"threads_started":1}
I20260806 04:59:37.933328 26873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:59:37.898335 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:41885 (local address 127.9.96.65:35167)
0806 04:59:37.910262 (+ 11927us) negotiation.cc:107] Waiting for socket to connect
0806 04:59:37.910335 (+ 73us) client_negotiation.cc:174] Beginning negotiation
0806 04:59:37.913207 (+ 2872us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:59:37.914456 (+ 1249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:59:37.914463 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:59:37.914896 (+ 433us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:59:37.915578 (+ 682us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:59:37.915604 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:59:37.917206 (+ 1602us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:59:37.917211 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:59:37.919165 (+ 1954us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:59:37.919190 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:59:37.920454 (+ 1264us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:59:37.920460 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:59:37.920732 (+ 272us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:59:37.921512 (+ 780us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:59:37.921541 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:59:37.924283 (+ 2742us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 04:59:37.928327 (+ 4044us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:59:37.928335 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:59:37.928351 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:59:37.928918 (+ 567us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:59:37.929363 (+ 445us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:59:37.929367 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:59:37.929371 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:59:37.929620 (+ 249us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:59:37.930375 (+ 755us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 04:59:37.930382 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 04:59:37.930756 (+ 374us) client_negotiation.cc:769] Sending connection context
0806 04:59:37.931047 (+ 291us) client_negotiation.cc:240] Negotiation successful
0806 04:59:37.931616 (+ 569us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11499,"thread_start_us":11262,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.96.66@KRBTEST.COM" created.
I20260806 04:59:37.935170 26871 heartbeater.cc:347] Connected to a master server at 127.9.96.126:41885
I20260806 04:59:37.935653 26871 heartbeater.cc:464] Registering TS with master...
I20260806 04:59:37.936547 26871 heartbeater.cc:511] Master 127.9.96.126:41885 requested a full tablet report, sending...
I20260806 04:59:37.939201 26531 ts_manager.cc:194] Registered new tserver with Master: f3d52b1be46b435ca76dfde46d3a93b2 (127.9.96.65:38267)
I20260806 04:59:37.941344 26531 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.96.65@KRBTEST.COM'} at 127.9.96.65:35167
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.96.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.66.keytab.
Entry for principal kudu/127.9.96.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.66.keytab.
I20260806 04:59:37.972893 9601 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
/tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785992227050689-9601-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785992227050689-9601-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785992227050689-9601-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785992227050689-9601-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.9.96.84:39207/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.96.66:0
--local_ip_for_outbound_sockets=127.9.96.66
--webserver_interface=127.9.96.66
--webserver_port=0
--tserver_master_addrs=127.9.96.126:41885
--keytab_file=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.66.keytab
--principal=kudu/127.9.96.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.96.84:43827
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785992227050689-9601-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785992227050689-9601-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785992227050689-9601-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 04:59:38 dist-test-slave-x4qh krb5kdc[25816](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992377, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.96.126@KRBTEST.COM for HTTP/127.9.96.84@KRBTEST.COM
W20260806 04:59:38.437338 26878 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 04:59:38.437637 26878 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 04:59:38.437814 26878 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 04:59:38.445124 26878 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 04:59:38.445271 26878 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 04:59:38.445436 26878 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.96.66
I20260806 04:59:38.455480 26878 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.96.84:43827
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785992227050689-9601-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785992227050689-9601-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.9.96.84:39207/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.66.keytab
--principal=kudu/127.9.96.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.96.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785992227050689-9601-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.96.66
--webserver_port=0
--tserver_master_addrs=127.9.96.126:41885
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.96.66
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785992227050689-9601-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 03:56:12 UTC on ccycloud.quasar-guijbm.root.comops.site
build id 4569618
ASAN enabled
I20260806 04:59:38.456692 26878 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 04:59:38.457986 26878 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 04:59:38.470741 26894 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:59:38.470682 26895 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:59:38.471899 26900 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:59:38.471884 26878 server_base.cc:1034] running on GCE node
I20260806 04:59:38.472733 26878 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 04:59:38.473881 26878 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 04:59:38.475127 26878 hybrid_clock.cc:648] HybridClock initialized: now 1785992378475050 us; error 77 us; skew 500 ppm
I20260806 04:59:38.475489 26878 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 04:59:38 dist-test-slave-x4qh krb5kdc[25816](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992378, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.96.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 04:59:38.479202 26878 init.cc:377] Logged in from keytab as kudu/127.9.96.66@KRBTEST.COM (short username kudu)
I20260806 04:59:38.480852 26878 webserver.cc:460] Webserver started at http://127.9.96.66:33707/ using document root <none> and password file <none>
I20260806 04:59:38.481494 26878 fs_manager.cc:359] Metadata directory not provided
I20260806 04:59:38.481616 26878 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 04:59:38.481937 26878 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 04:59:38 dist-test-slave-x4qh krb5kdc[25816](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992378, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.96.66@KRBTEST.COM for HTTP/127.9.96.84@KRBTEST.COM
I20260806 04:59:38.496279 26878 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785992227050689-9601-0/minicluster-data/ts-1/data/instance:
uuid: "00a49322e3754e149658cbab791e9b00"
format_stamp: "Formatted at 2026-08-06 04:59:38 on dist-test-slave-x4qh"
server_key: "940cadf71235ef4303cca1f254ba3b86"
server_key_iv: "9043754d87f2336afaeb614b0e7589a3"
server_key_version: "kuduclusterkey@0"
I20260806 04:59:38.497242 26878 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785992227050689-9601-0/minicluster-data/ts-1/wal/instance:
uuid: "00a49322e3754e149658cbab791e9b00"
format_stamp: "Formatted at 2026-08-06 04:59:38 on dist-test-slave-x4qh"
server_key: "940cadf71235ef4303cca1f254ba3b86"
server_key_iv: "9043754d87f2336afaeb614b0e7589a3"
server_key_version: "kuduclusterkey@0"
I20260806 04:59:38.502661 26878 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260806 04:59:38.518886 26911 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 04:59:38.520725 26878 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260806 04:59:38.520946 26878 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785992227050689-9601-0/minicluster-data/ts-1/data,/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785992227050689-9601-0/minicluster-data/ts-1/wal
uuid: "00a49322e3754e149658cbab791e9b00"
format_stamp: "Formatted at 2026-08-06 04:59:38 on dist-test-slave-x4qh"
server_key: "940cadf71235ef4303cca1f254ba3b86"
server_key_iv: "9043754d87f2336afaeb614b0e7589a3"
server_key_version: "kuduclusterkey@0"
I20260806 04:59:38.521222 26878 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785992227050689-9601-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785992227050689-9601-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785992227050689-9601-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 04:59:38.533476 26878 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 04:59:38.537343 26878 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 04:59:38.537645 26878 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 04:59:38.538568 26878 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 04:59:38.539963 26878 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 04:59:38.540066 26878 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 04:59:38.540210 26878 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 04:59:38.540285 26878 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 04:59:38.585886 26878 rpc_server.cc:307] RPC server started. Bound to: 127.9.96.66:40351
I20260806 04:59:38.585906 27086 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.96.66:40351 every 8 connection(s)
I20260806 04:59:38.587311 26878 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785992227050689-9601-0/minicluster-data/ts-1/data/info.pb
I20260806 04:59:38.594270 9601 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu as pid 26878
I20260806 04:59:38.594463 9601 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785992227050689-9601-0/minicluster-data/ts-1/wal/instance
I20260806 04:59:38.594918 9601 external_mini_cluster.cc:1428] Setting key be2687dd381fc56929e68bd87e9011ac
Aug 06 04:59:38 dist-test-slave-x4qh krb5kdc[25816](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992378, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.96.66@KRBTEST.COM for kudu/127.9.96.126@KRBTEST.COM
I20260806 04:59:38.611840 27093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:59:38.591535 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.96.66:56021 (local address 127.9.96.126:41885)
0806 04:59:38.592050 (+ 515us) server_negotiation.cc:206] Beginning negotiation
0806 04:59:38.592056 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 04:59:38.592077 (+ 21us) server_negotiation.cc:407] Connection header received
0806 04:59:38.593790 (+ 1713us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:59:38.593796 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:59:38.593919 (+ 123us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:59:38.594048 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:59:38.595991 (+ 1943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:59:38.597182 (+ 1191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:59:38.599786 (+ 2604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:59:38.600487 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:59:38.600631 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:59:38.605367 (+ 4736us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:59:38.605375 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:59:38.605380 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:59:38.605424 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:59:38.608831 (+ 3407us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:59:38.609581 (+ 750us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:59:38.609586 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:59:38.609589 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:59:38.609725 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:59:38.610188 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:59:38.610194 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:59:38.610202 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:59:38.610467 (+ 265us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:59:38.610633 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0806 04:59:38.611326 (+ 693us) server_negotiation.cc:299] Negotiation successful
0806 04:59:38.611571 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":379,"thread_start_us":903,"threads_started":1}
I20260806 04:59:38.613411 27091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:59:38.590939 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:41885 (local address 127.9.96.66:56021)
0806 04:59:38.591898 (+ 959us) negotiation.cc:107] Waiting for socket to connect
0806 04:59:38.591957 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0806 04:59:38.593527 (+ 1570us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:59:38.594328 (+ 801us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:59:38.594340 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:59:38.594939 (+ 599us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:59:38.595757 (+ 818us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:59:38.595783 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:59:38.597375 (+ 1592us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:59:38.597381 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:59:38.599612 (+ 2231us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:59:38.599630 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:59:38.600746 (+ 1116us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:59:38.600756 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:59:38.601115 (+ 359us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:59:38.601960 (+ 845us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:59:38.602008 (+ 48us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:59:38.605088 (+ 3080us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 04:59:38.608968 (+ 3880us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:59:38.608978 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:59:38.608996 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:59:38.609450 (+ 454us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:59:38.609838 (+ 388us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:59:38.609843 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:59:38.609847 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:59:38.610057 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:59:38.610607 (+ 550us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 04:59:38.610614 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 04:59:38.610929 (+ 315us) client_negotiation.cc:769] Sending connection context
0806 04:59:38.612059 (+ 1130us) client_negotiation.cc:240] Negotiation successful
0806 04:59:38.612475 (+ 416us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":627,"thread_start_us":639,"threads_started":1}
I20260806 04:59:38.615007 27087 heartbeater.cc:347] Connected to a master server at 127.9.96.126:41885
I20260806 04:59:38.615345 27087 heartbeater.cc:464] Registering TS with master...
I20260806 04:59:38.616127 27087 heartbeater.cc:511] Master 127.9.96.126:41885 requested a full tablet report, sending...
I20260806 04:59:38.617708 26531 ts_manager.cc:194] Registered new tserver with Master: 00a49322e3754e149658cbab791e9b00 (127.9.96.66:40351)
I20260806 04:59:38.618683 26531 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.96.66@KRBTEST.COM'} at 127.9.96.66:56021
WARNING: no policy specified for kudu/127.9.96.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.96.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.96.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.67.keytab.
Entry for principal kudu/127.9.96.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.67.keytab.
I20260806 04:59:38.663856 9601 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
/tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785992227050689-9601-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785992227050689-9601-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785992227050689-9601-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785992227050689-9601-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.9.96.84:39207/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.96.67:0
--local_ip_for_outbound_sockets=127.9.96.67
--webserver_interface=127.9.96.67
--webserver_port=0
--tserver_master_addrs=127.9.96.126:41885
--keytab_file=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.67.keytab
--principal=kudu/127.9.96.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.96.84:43827
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785992227050689-9601-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785992227050689-9601-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785992227050689-9601-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 04:59:38.945194 26871 heartbeater.cc:503] Master 127.9.96.126:41885 was elected leader, sending a full tablet report...
W20260806 04:59:39.015484 27097 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 04:59:39.015755 27097 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 04:59:39.015930 27097 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 04:59:39.023422 27097 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 04:59:39.023535 27097 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 04:59:39.023741 27097 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.96.67
I20260806 04:59:39.033789 27097 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.96.84:43827
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785992227050689-9601-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785992227050689-9601-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.9.96.84:39207/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.67.keytab
--principal=kudu/127.9.96.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.96.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785992227050689-9601-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.96.67
--webserver_port=0
--tserver_master_addrs=127.9.96.126:41885
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.96.67
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785992227050689-9601-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 03:56:12 UTC on ccycloud.quasar-guijbm.root.comops.site
build id 4569618
ASAN enabled
I20260806 04:59:39.034991 27097 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 04:59:39.036307 27097 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 04:59:39.049401 27112 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:59:39.049484 27109 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:59:39.049472 27106 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:59:39.050140 27097 server_base.cc:1034] running on GCE node
I20260806 04:59:39.050688 27097 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 04:59:39.051738 27097 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 04:59:39.052977 27097 hybrid_clock.cc:648] HybridClock initialized: now 1785992379052948 us; error 52 us; skew 500 ppm
I20260806 04:59:39.053311 27097 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 04:59:39 dist-test-slave-x4qh krb5kdc[25816](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992379, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.96.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 04:59:39.056411 27097 init.cc:377] Logged in from keytab as kudu/127.9.96.67@KRBTEST.COM (short username kudu)
I20260806 04:59:39.058032 27097 webserver.cc:460] Webserver started at http://127.9.96.67:43963/ using document root <none> and password file <none>
I20260806 04:59:39.058961 27097 fs_manager.cc:359] Metadata directory not provided
I20260806 04:59:39.059162 27097 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 04:59:39.059690 27097 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 04:59:39 dist-test-slave-x4qh krb5kdc[25816](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992379, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.96.67@KRBTEST.COM for HTTP/127.9.96.84@KRBTEST.COM
I20260806 04:59:39.079159 27097 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785992227050689-9601-0/minicluster-data/ts-2/data/instance:
uuid: "f02994845a3c42eea4de4b9efd850186"
format_stamp: "Formatted at 2026-08-06 04:59:39 on dist-test-slave-x4qh"
server_key: "de230c5d67c46569774ac015965ccc19"
server_key_iv: "682c4b7038fd3b078066a2795ef83c57"
server_key_version: "kuduclusterkey@0"
I20260806 04:59:39.080003 27097 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785992227050689-9601-0/minicluster-data/ts-2/wal/instance:
uuid: "f02994845a3c42eea4de4b9efd850186"
format_stamp: "Formatted at 2026-08-06 04:59:39 on dist-test-slave-x4qh"
server_key: "de230c5d67c46569774ac015965ccc19"
server_key_iv: "682c4b7038fd3b078066a2795ef83c57"
server_key_version: "kuduclusterkey@0"
I20260806 04:59:39.085675 27097 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260806 04:59:39.101842 27125 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 04:59:39.103303 27097 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.001s
I20260806 04:59:39.103504 27097 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785992227050689-9601-0/minicluster-data/ts-2/data,/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785992227050689-9601-0/minicluster-data/ts-2/wal
uuid: "f02994845a3c42eea4de4b9efd850186"
format_stamp: "Formatted at 2026-08-06 04:59:39 on dist-test-slave-x4qh"
server_key: "de230c5d67c46569774ac015965ccc19"
server_key_iv: "682c4b7038fd3b078066a2795ef83c57"
server_key_version: "kuduclusterkey@0"
I20260806 04:59:39.103720 27097 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785992227050689-9601-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785992227050689-9601-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785992227050689-9601-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 04:59:39.125697 27097 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 04:59:39.129297 27097 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 04:59:39.129554 27097 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 04:59:39.130374 27097 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 04:59:39.131770 27097 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 04:59:39.131850 27097 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 04:59:39.131932 27097 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 04:59:39.131994 27097 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 04:59:39.163977 27097 rpc_server.cc:307] RPC server started. Bound to: 127.9.96.67:38665
I20260806 04:59:39.164007 27307 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.96.67:38665 every 8 connection(s)
I20260806 04:59:39.165225 27097 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785992227050689-9601-0/minicluster-data/ts-2/data/info.pb
I20260806 04:59:39.173283 9601 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu as pid 27097
I20260806 04:59:39.173480 9601 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785992227050689-9601-0/minicluster-data/ts-2/wal/instance
I20260806 04:59:39.173928 9601 external_mini_cluster.cc:1428] Setting key f40926774dee4f435d60ea3fbc76e633
Aug 06 04:59:39 dist-test-slave-x4qh krb5kdc[25816](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992379, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.96.67@KRBTEST.COM for kudu/127.9.96.126@KRBTEST.COM
I20260806 04:59:39.184696 27312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:59:39.167777 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.96.67:33555 (local address 127.9.96.126:41885)
0806 04:59:39.168283 (+ 506us) server_negotiation.cc:206] Beginning negotiation
0806 04:59:39.168288 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 04:59:39.169196 (+ 908us) server_negotiation.cc:407] Connection header received
0806 04:59:39.170863 (+ 1667us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:59:39.170867 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:59:39.170935 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:59:39.171033 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:59:39.172421 (+ 1388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:59:39.173122 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:59:39.175072 (+ 1950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:59:39.175773 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:59:39.175876 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:59:39.179304 (+ 3428us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:59:39.179309 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:59:39.179312 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:59:39.179344 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:59:39.181933 (+ 2589us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:59:39.182645 (+ 712us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:59:39.182649 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:59:39.182651 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:59:39.182766 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:59:39.183140 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:59:39.183143 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:59:39.183145 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:59:39.183458 (+ 313us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:59:39.183573 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0806 04:59:39.184247 (+ 674us) server_negotiation.cc:299] Negotiation successful
0806 04:59:39.184449 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":396,"thread_start_us":237,"threads_started":1}
I20260806 04:59:39.185456 27313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:59:39.168315 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:41885 (local address 127.9.96.67:33555)
0806 04:59:39.169029 (+ 714us) negotiation.cc:107] Waiting for socket to connect
0806 04:59:39.169071 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0806 04:59:39.170597 (+ 1526us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:59:39.171223 (+ 626us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:59:39.171230 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:59:39.171718 (+ 488us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:59:39.172265 (+ 547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:59:39.172284 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:59:39.173293 (+ 1009us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:59:39.173297 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:59:39.174929 (+ 1632us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:59:39.174944 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:59:39.176028 (+ 1084us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:59:39.176037 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:59:39.176358 (+ 321us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:59:39.177107 (+ 749us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:59:39.177144 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:59:39.179117 (+ 1973us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 04:59:39.182089 (+ 2972us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:59:39.182095 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:59:39.182106 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:59:39.182552 (+ 446us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:59:39.182859 (+ 307us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:59:39.182861 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:59:39.182863 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:59:39.183019 (+ 156us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:59:39.183567 (+ 548us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 04:59:39.183572 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 04:59:39.183829 (+ 257us) client_negotiation.cc:769] Sending connection context
0806 04:59:39.184044 (+ 215us) client_negotiation.cc:240] Negotiation successful
0806 04:59:39.184405 (+ 361us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":508,"thread_start_us":302,"threads_started":1}
I20260806 04:59:39.186755 27308 heartbeater.cc:347] Connected to a master server at 127.9.96.126:41885
I20260806 04:59:39.187014 27308 heartbeater.cc:464] Registering TS with master...
I20260806 04:59:39.187744 27308 heartbeater.cc:511] Master 127.9.96.126:41885 requested a full tablet report, sending...
I20260806 04:59:39.189267 26530 ts_manager.cc:194] Registered new tserver with Master: f02994845a3c42eea4de4b9efd850186 (127.9.96.67:38665)
I20260806 04:59:39.190150 26530 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.96.67@KRBTEST.COM'} at 127.9.96.67:33555
I20260806 04:59:39.201346 9601 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 04:59:39.218744 27312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:59:39.205977 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58494 (local address 127.9.96.126:41885)
0806 04:59:39.206125 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0806 04:59:39.206129 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 04:59:39.206670 (+ 541us) server_negotiation.cc:407] Connection header received
0806 04:59:39.206806 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:59:39.206809 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:59:39.206935 (+ 126us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:59:39.207029 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:59:39.208630 (+ 1601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:59:39.209431 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:59:39.210841 (+ 1410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:59:39.211236 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:59:39.211345 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:59:39.213774 (+ 2429us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:59:39.213778 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:59:39.213781 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:59:39.213811 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:59:39.216430 (+ 2619us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:59:39.217049 (+ 619us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:59:39.217053 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:59:39.217055 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:59:39.217224 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:59:39.217570 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:59:39.217573 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:59:39.217575 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:59:39.217826 (+ 251us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:59:39.218009 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0806 04:59:39.218343 (+ 334us) server_negotiation.cc:299] Negotiation successful
0806 04:59:39.218527 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260806 04:59:39.222556 26530 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58494:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 04:59:39.229817 26530 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 04:59:39.259891 27329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:59:39.243139 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57660 (local address 127.9.96.66:40351)
0806 04:59:39.248594 (+ 5455us) server_negotiation.cc:206] Beginning negotiation
0806 04:59:39.248605 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 04:59:39.248638 (+ 33us) server_negotiation.cc:407] Connection header received
0806 04:59:39.248751 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:59:39.248761 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:59:39.248993 (+ 232us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:59:39.249225 (+ 232us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 04:59:39.250053 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:59:39.252269 (+ 2216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:59:39.256738 (+ 4469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:59:39.258305 (+ 1567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:59:39.258445 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:59:39.258633 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0806 04:59:39.259400 (+ 767us) server_negotiation.cc:299] Negotiation successful
0806 04:59:39.259576 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5267,"thread_start_us":5138,"threads_started":1}
I20260806 04:59:39.260042 27328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:59:39.242148 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.67:38665 (local address 127.0.0.1:37054)
0806 04:59:39.244494 (+ 2346us) negotiation.cc:107] Waiting for socket to connect
0806 04:59:39.244536 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0806 04:59:39.244703 (+ 167us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:59:39.250601 (+ 5898us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:59:39.250609 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:59:39.250631 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 04:59:39.251038 (+ 407us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:59:39.251055 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:59:39.253351 (+ 2296us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:59:39.253359 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:59:39.256148 (+ 2789us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:59:39.256163 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:59:39.259148 (+ 2985us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:59:39.259154 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:59:39.259339 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:59:39.259368 (+ 29us) client_negotiation.cc:769] Sending connection context
0806 04:59:39.259452 (+ 84us) client_negotiation.cc:240] Negotiation successful
0806 04:59:39.259633 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1848,"mutex_wait_us":151,"thread_start_us":8058,"threads_started":1}
I20260806 04:59:39.261003 27332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:59:39.244590 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50030 (local address 127.9.96.65:38267)
0806 04:59:39.249188 (+ 4598us) server_negotiation.cc:206] Beginning negotiation
0806 04:59:39.249200 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 04:59:39.250641 (+ 1441us) server_negotiation.cc:407] Connection header received
0806 04:59:39.250752 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:59:39.250760 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:59:39.251021 (+ 261us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:59:39.251205 (+ 184us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 04:59:39.252190 (+ 985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:59:39.254085 (+ 1895us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:59:39.257417 (+ 3332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:59:39.259065 (+ 1648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:59:39.259218 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:59:39.259510 (+ 292us) server_negotiation.cc:1035] Waiting for connection context
0806 04:59:39.260483 (+ 973us) server_negotiation.cc:299] Negotiation successful
0806 04:59:39.260649 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4383,"thread_start_us":4199,"threads_started":1}
I20260806 04:59:39.261204 27327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:59:39.242359 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37054 (local address 127.9.96.67:38665)
0806 04:59:39.249977 (+ 7618us) server_negotiation.cc:206] Beginning negotiation
0806 04:59:39.249987 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 04:59:39.250022 (+ 35us) server_negotiation.cc:407] Connection header received
0806 04:59:39.250122 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:59:39.250131 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:59:39.250407 (+ 276us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:59:39.251223 (+ 816us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 04:59:39.251583 (+ 360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:59:39.253192 (+ 1609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:59:39.257415 (+ 4223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:59:39.258843 (+ 1428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:59:39.260451 (+ 1608us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:59:39.260661 (+ 210us) server_negotiation.cc:1035] Waiting for connection context
0806 04:59:39.260722 (+ 61us) server_negotiation.cc:299] Negotiation successful
0806 04:59:39.260883 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7451,"thread_start_us":7300,"threads_started":1}
I20260806 04:59:39.261786 27326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:59:39.244441 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.65:38267 (local address 127.0.0.1:50030)
0806 04:59:39.250388 (+ 5947us) negotiation.cc:107] Waiting for socket to connect
0806 04:59:39.250404 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 04:59:39.250526 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:59:39.251519 (+ 993us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:59:39.251524 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:59:39.251553 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 04:59:39.251977 (+ 424us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:59:39.251990 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:59:39.254300 (+ 2310us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:59:39.254305 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:59:39.257187 (+ 2882us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:59:39.257201 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:59:39.259677 (+ 2476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:59:39.259685 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:59:39.259909 (+ 224us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:59:39.259953 (+ 44us) client_negotiation.cc:769] Sending connection context
0806 04:59:39.260049 (+ 96us) client_negotiation.cc:240] Negotiation successful
0806 04:59:39.261283 (+ 1234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5802,"mutex_wait_us":48,"thread_start_us":3129,"threads_started":1}
I20260806 04:59:39.262205 27330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:59:39.242503 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.66:40351 (local address 127.0.0.1:57660)
0806 04:59:39.247892 (+ 5389us) negotiation.cc:107] Waiting for socket to connect
0806 04:59:39.247913 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0806 04:59:39.248041 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:59:39.249356 (+ 1315us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:59:39.249364 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:59:39.249406 (+ 42us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 04:59:39.249826 (+ 420us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:59:39.249839 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:59:39.253568 (+ 3729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:59:39.253573 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:59:39.256568 (+ 2995us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:59:39.256580 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:59:39.258819 (+ 2239us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:59:39.258825 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:59:39.259050 (+ 225us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:59:39.259139 (+ 89us) client_negotiation.cc:769] Sending connection context
0806 04:59:39.260240 (+ 1101us) client_negotiation.cc:240] Negotiation successful
0806 04:59:39.261610 (+ 1370us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5237,"mutex_wait_us":1371,"thread_start_us":1365,"threads_started":1}
I20260806 04:59:39.265565 27200 tablet_service.cc:1467] Processing CreateTablet for tablet 71c17fc89643456a982ffedda01ed1d2 (DEFAULT_TABLE table=test-table [id=112e77eefa6443baae1a12d00fb9a47e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 04:59:39.266225 26988 tablet_service.cc:1467] Processing CreateTablet for tablet 71c17fc89643456a982ffedda01ed1d2 (DEFAULT_TABLE table=test-table [id=112e77eefa6443baae1a12d00fb9a47e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 04:59:39.267489 27200 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 71c17fc89643456a982ffedda01ed1d2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 04:59:39.267647 26988 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 71c17fc89643456a982ffedda01ed1d2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 04:59:39.268000 26765 tablet_service.cc:1467] Processing CreateTablet for tablet 71c17fc89643456a982ffedda01ed1d2 (DEFAULT_TABLE table=test-table [id=112e77eefa6443baae1a12d00fb9a47e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 04:59:39.269639 26765 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 71c17fc89643456a982ffedda01ed1d2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 04:59:39.282141 27334 tablet_bootstrap.cc:492] T 71c17fc89643456a982ffedda01ed1d2 P 00a49322e3754e149658cbab791e9b00: Bootstrap starting.
I20260806 04:59:39.287041 27334 tablet_bootstrap.cc:654] T 71c17fc89643456a982ffedda01ed1d2 P 00a49322e3754e149658cbab791e9b00: Neither blocks nor log segments found. Creating new log.
I20260806 04:59:39.288475 27334 log.cc:824] T 71c17fc89643456a982ffedda01ed1d2 P 00a49322e3754e149658cbab791e9b00: Log is configured to *not* fsync() on all Append() calls
I20260806 04:59:39.288756 27336 tablet_bootstrap.cc:492] T 71c17fc89643456a982ffedda01ed1d2 P f3d52b1be46b435ca76dfde46d3a93b2: Bootstrap starting.
I20260806 04:59:39.294523 27336 tablet_bootstrap.cc:654] T 71c17fc89643456a982ffedda01ed1d2 P f3d52b1be46b435ca76dfde46d3a93b2: Neither blocks nor log segments found. Creating new log.
I20260806 04:59:39.295909 27336 log.cc:824] T 71c17fc89643456a982ffedda01ed1d2 P f3d52b1be46b435ca76dfde46d3a93b2: Log is configured to *not* fsync() on all Append() calls
I20260806 04:59:39.296406 27335 tablet_bootstrap.cc:492] T 71c17fc89643456a982ffedda01ed1d2 P f02994845a3c42eea4de4b9efd850186: Bootstrap starting.
I20260806 04:59:39.297924 27334 tablet_bootstrap.cc:492] T 71c17fc89643456a982ffedda01ed1d2 P 00a49322e3754e149658cbab791e9b00: No bootstrap required, opened a new log
I20260806 04:59:39.298203 27334 ts_tablet_manager.cc:1397] T 71c17fc89643456a982ffedda01ed1d2 P 00a49322e3754e149658cbab791e9b00: Time spent bootstrapping tablet: real 0.016s user 0.006s sys 0.006s
I20260806 04:59:39.300679 27335 tablet_bootstrap.cc:654] T 71c17fc89643456a982ffedda01ed1d2 P f02994845a3c42eea4de4b9efd850186: Neither blocks nor log segments found. Creating new log.
I20260806 04:59:39.302162 27335 log.cc:824] T 71c17fc89643456a982ffedda01ed1d2 P f02994845a3c42eea4de4b9efd850186: Log is configured to *not* fsync() on all Append() calls
I20260806 04:59:39.313678 27336 tablet_bootstrap.cc:492] T 71c17fc89643456a982ffedda01ed1d2 P f3d52b1be46b435ca76dfde46d3a93b2: No bootstrap required, opened a new log
I20260806 04:59:39.313484 27334 raft_consensus.cc:348] T 71c17fc89643456a982ffedda01ed1d2 P 00a49322e3754e149658cbab791e9b00 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f02994845a3c42eea4de4b9efd850186" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 38665 } } peers { permanent_uuid: "00a49322e3754e149658cbab791e9b00" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 40351 } } peers { permanent_uuid: "f3d52b1be46b435ca76dfde46d3a93b2" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 38267 } }
I20260806 04:59:39.313982 27334 raft_consensus.cc:374] T 71c17fc89643456a982ffedda01ed1d2 P 00a49322e3754e149658cbab791e9b00 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 04:59:39.314015 27336 ts_tablet_manager.cc:1397] T 71c17fc89643456a982ffedda01ed1d2 P f3d52b1be46b435ca76dfde46d3a93b2: Time spent bootstrapping tablet: real 0.025s user 0.010s sys 0.011s
I20260806 04:59:39.314088 27334 raft_consensus.cc:729] T 71c17fc89643456a982ffedda01ed1d2 P 00a49322e3754e149658cbab791e9b00 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 00a49322e3754e149658cbab791e9b00, State: Initialized, Role: FOLLOWER
I20260806 04:59:39.314697 27334 consensus_queue.cc:260] T 71c17fc89643456a982ffedda01ed1d2 P 00a49322e3754e149658cbab791e9b00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f02994845a3c42eea4de4b9efd850186" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 38665 } } peers { permanent_uuid: "00a49322e3754e149658cbab791e9b00" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 40351 } } peers { permanent_uuid: "f3d52b1be46b435ca76dfde46d3a93b2" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 38267 } }
I20260806 04:59:39.316165 27335 tablet_bootstrap.cc:492] T 71c17fc89643456a982ffedda01ed1d2 P f02994845a3c42eea4de4b9efd850186: No bootstrap required, opened a new log
I20260806 04:59:39.316463 27335 ts_tablet_manager.cc:1397] T 71c17fc89643456a982ffedda01ed1d2 P f02994845a3c42eea4de4b9efd850186: Time spent bootstrapping tablet: real 0.020s user 0.006s sys 0.011s
I20260806 04:59:39.325598 27087 heartbeater.cc:503] Master 127.9.96.126:41885 was elected leader, sending a full tablet report...
I20260806 04:59:39.326500 27334 ts_tablet_manager.cc:1428] T 71c17fc89643456a982ffedda01ed1d2 P 00a49322e3754e149658cbab791e9b00: Time spent starting tablet: real 0.028s user 0.027s sys 0.002s
I20260806 04:59:39.325024 27336 raft_consensus.cc:348] T 71c17fc89643456a982ffedda01ed1d2 P f3d52b1be46b435ca76dfde46d3a93b2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f02994845a3c42eea4de4b9efd850186" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 38665 } } peers { permanent_uuid: "00a49322e3754e149658cbab791e9b00" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 40351 } } peers { permanent_uuid: "f3d52b1be46b435ca76dfde46d3a93b2" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 38267 } }
I20260806 04:59:39.327139 27336 raft_consensus.cc:374] T 71c17fc89643456a982ffedda01ed1d2 P f3d52b1be46b435ca76dfde46d3a93b2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 04:59:39.327293 27336 raft_consensus.cc:729] T 71c17fc89643456a982ffedda01ed1d2 P f3d52b1be46b435ca76dfde46d3a93b2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3d52b1be46b435ca76dfde46d3a93b2, State: Initialized, Role: FOLLOWER
I20260806 04:59:39.328035 27336 consensus_queue.cc:260] T 71c17fc89643456a982ffedda01ed1d2 P f3d52b1be46b435ca76dfde46d3a93b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f02994845a3c42eea4de4b9efd850186" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 38665 } } peers { permanent_uuid: "00a49322e3754e149658cbab791e9b00" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 40351 } } peers { permanent_uuid: "f3d52b1be46b435ca76dfde46d3a93b2" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 38267 } }
I20260806 04:59:39.328567 27335 raft_consensus.cc:348] T 71c17fc89643456a982ffedda01ed1d2 P f02994845a3c42eea4de4b9efd850186 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f02994845a3c42eea4de4b9efd850186" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 38665 } } peers { permanent_uuid: "00a49322e3754e149658cbab791e9b00" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 40351 } } peers { permanent_uuid: "f3d52b1be46b435ca76dfde46d3a93b2" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 38267 } }
I20260806 04:59:39.329037 27335 raft_consensus.cc:374] T 71c17fc89643456a982ffedda01ed1d2 P f02994845a3c42eea4de4b9efd850186 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 04:59:39.329183 27335 raft_consensus.cc:729] T 71c17fc89643456a982ffedda01ed1d2 P f02994845a3c42eea4de4b9efd850186 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f02994845a3c42eea4de4b9efd850186, State: Initialized, Role: FOLLOWER
I20260806 04:59:39.329931 27335 consensus_queue.cc:260] T 71c17fc89643456a982ffedda01ed1d2 P f02994845a3c42eea4de4b9efd850186 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f02994845a3c42eea4de4b9efd850186" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 38665 } } peers { permanent_uuid: "00a49322e3754e149658cbab791e9b00" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 40351 } } peers { permanent_uuid: "f3d52b1be46b435ca76dfde46d3a93b2" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 38267 } }
I20260806 04:59:39.334936 27335 ts_tablet_manager.cc:1428] T 71c17fc89643456a982ffedda01ed1d2 P f02994845a3c42eea4de4b9efd850186: Time spent starting tablet: real 0.018s user 0.002s sys 0.014s
I20260806 04:59:39.335289 27308 heartbeater.cc:503] Master 127.9.96.126:41885 was elected leader, sending a full tablet report...
I20260806 04:59:39.335300 27336 ts_tablet_manager.cc:1428] T 71c17fc89643456a982ffedda01ed1d2 P f3d52b1be46b435ca76dfde46d3a93b2: Time spent starting tablet: real 0.021s user 0.015s sys 0.004s
W20260806 04:59:39.339601 27089 tablet.cc:2369] T 71c17fc89643456a982ffedda01ed1d2 P 00a49322e3754e149658cbab791e9b00: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 04:59:39.398952 26872 tablet.cc:2369] T 71c17fc89643456a982ffedda01ed1d2 P f3d52b1be46b435ca76dfde46d3a93b2: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 04:59:39.416955 27310 tablet.cc:2369] T 71c17fc89643456a982ffedda01ed1d2 P f02994845a3c42eea4de4b9efd850186: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 04:59:39.445230 27346 raft_consensus.cc:482] T 71c17fc89643456a982ffedda01ed1d2 P f02994845a3c42eea4de4b9efd850186 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 04:59:39.445574 27346 raft_consensus.cc:504] T 71c17fc89643456a982ffedda01ed1d2 P f02994845a3c42eea4de4b9efd850186 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f02994845a3c42eea4de4b9efd850186" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 38665 } } peers { permanent_uuid: "00a49322e3754e149658cbab791e9b00" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 40351 } } peers { permanent_uuid: "f3d52b1be46b435ca76dfde46d3a93b2" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 38267 } }
I20260806 04:59:39.448086 27346 leader_election.cc:288] T 71c17fc89643456a982ffedda01ed1d2 P f02994845a3c42eea4de4b9efd850186 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 00a49322e3754e149658cbab791e9b00 (127.9.96.66:40351), f3d52b1be46b435ca76dfde46d3a93b2 (127.9.96.65:38267)
I20260806 04:59:39.455243 27332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:59:39.448732 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.96.67:43009 (local address 127.9.96.65:38267)
0806 04:59:39.448935 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0806 04:59:39.448942 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 04:59:39.448964 (+ 22us) server_negotiation.cc:407] Connection header received
0806 04:59:39.449050 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:59:39.449055 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:59:39.449131 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:59:39.449246 (+ 115us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 04:59:39.449995 (+ 749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:59:39.451344 (+ 1349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:59:39.453698 (+ 2354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:59:39.454425 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:59:39.454557 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:59:39.454652 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0806 04:59:39.454898 (+ 246us) server_negotiation.cc:299] Negotiation successful
0806 04:59:39.455012 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260806 04:59:39.455240 27313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:59:39.448319 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.65:38267 (local address 127.9.96.67:43009)
0806 04:59:39.448601 (+ 282us) negotiation.cc:107] Waiting for socket to connect
0806 04:59:39.448624 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0806 04:59:39.448770 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:59:39.449502 (+ 732us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:59:39.449510 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:59:39.449546 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 04:59:39.449790 (+ 244us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:59:39.449806 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:59:39.451629 (+ 1823us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:59:39.451637 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:59:39.453559 (+ 1922us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:59:39.453572 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:59:39.454572 (+ 1000us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:59:39.454577 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:59:39.454745 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:59:39.454767 (+ 22us) client_negotiation.cc:769] Sending connection context
0806 04:59:39.454865 (+ 98us) client_negotiation.cc:240] Negotiation successful
0806 04:59:39.454985 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151,"mutex_wait_us":39}
I20260806 04:59:39.456291 27329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:59:39.449057 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.96.67:37833 (local address 127.9.96.66:40351)
0806 04:59:39.449508 (+ 451us) server_negotiation.cc:206] Beginning negotiation
0806 04:59:39.449514 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 04:59:39.449533 (+ 19us) server_negotiation.cc:407] Connection header received
0806 04:59:39.449662 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:59:39.449665 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:59:39.449732 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:59:39.449829 (+ 97us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 04:59:39.450315 (+ 486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:59:39.451039 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:59:39.453616 (+ 2577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:59:39.455042 (+ 1426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:59:39.455195 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:59:39.455304 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0806 04:59:39.455881 (+ 577us) server_negotiation.cc:299] Negotiation successful
0806 04:59:39.456019 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":293}
I20260806 04:59:39.456480 27347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:59:39.448383 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.66:40351 (local address 127.9.96.67:37833)
0806 04:59:39.449397 (+ 1014us) negotiation.cc:107] Waiting for socket to connect
0806 04:59:39.449415 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 04:59:39.449529 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:59:39.449845 (+ 316us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:59:39.449849 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:59:39.449882 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 04:59:39.450172 (+ 290us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:59:39.450179 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:59:39.451172 (+ 993us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:59:39.451182 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:59:39.453436 (+ 2254us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:59:39.453450 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:59:39.455485 (+ 2035us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:59:39.455495 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:59:39.455700 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:59:39.455743 (+ 43us) client_negotiation.cc:769] Sending connection context
0806 04:59:39.455846 (+ 103us) client_negotiation.cc:240] Negotiation successful
0806 04:59:39.455994 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":898,"mutex_wait_us":63,"thread_start_us":705,"threads_started":1}
I20260806 04:59:39.456451 26796 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "71c17fc89643456a982ffedda01ed1d2" candidate_uuid: "f02994845a3c42eea4de4b9efd850186" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3d52b1be46b435ca76dfde46d3a93b2" is_pre_election: true
I20260806 04:59:39.457080 26796 raft_consensus.cc:2454] T 71c17fc89643456a982ffedda01ed1d2 P f3d52b1be46b435ca76dfde46d3a93b2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f02994845a3c42eea4de4b9efd850186 in term 0.
I20260806 04:59:39.457782 27128 leader_election.cc:302] T 71c17fc89643456a982ffedda01ed1d2 P f02994845a3c42eea4de4b9efd850186 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f02994845a3c42eea4de4b9efd850186, f3d52b1be46b435ca76dfde46d3a93b2; no voters:
I20260806 04:59:39.457680 27017 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "71c17fc89643456a982ffedda01ed1d2" candidate_uuid: "f02994845a3c42eea4de4b9efd850186" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "00a49322e3754e149658cbab791e9b00" is_pre_election: true
I20260806 04:59:39.458139 27346 raft_consensus.cc:2781] T 71c17fc89643456a982ffedda01ed1d2 P f02994845a3c42eea4de4b9efd850186 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 04:59:39.458287 27346 raft_consensus.cc:482] T 71c17fc89643456a982ffedda01ed1d2 P f02994845a3c42eea4de4b9efd850186 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 04:59:39.458427 27346 raft_consensus.cc:3037] T 71c17fc89643456a982ffedda01ed1d2 P f02994845a3c42eea4de4b9efd850186 [term 0 FOLLOWER]: Advancing to term 1
I20260806 04:59:39.458263 27017 raft_consensus.cc:2454] T 71c17fc89643456a982ffedda01ed1d2 P 00a49322e3754e149658cbab791e9b00 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f02994845a3c42eea4de4b9efd850186 in term 0.
I20260806 04:59:39.460784 27346 raft_consensus.cc:504] T 71c17fc89643456a982ffedda01ed1d2 P f02994845a3c42eea4de4b9efd850186 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f02994845a3c42eea4de4b9efd850186" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 38665 } } peers { permanent_uuid: "00a49322e3754e149658cbab791e9b00" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 40351 } } peers { permanent_uuid: "f3d52b1be46b435ca76dfde46d3a93b2" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 38267 } }
I20260806 04:59:39.461421 27346 leader_election.cc:288] T 71c17fc89643456a982ffedda01ed1d2 P f02994845a3c42eea4de4b9efd850186 [CANDIDATE]: Term 1 election: Requested vote from peers 00a49322e3754e149658cbab791e9b00 (127.9.96.66:40351), f3d52b1be46b435ca76dfde46d3a93b2 (127.9.96.65:38267)
I20260806 04:59:39.461877 26796 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "71c17fc89643456a982ffedda01ed1d2" candidate_uuid: "f02994845a3c42eea4de4b9efd850186" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3d52b1be46b435ca76dfde46d3a93b2"
I20260806 04:59:39.461923 27017 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "71c17fc89643456a982ffedda01ed1d2" candidate_uuid: "f02994845a3c42eea4de4b9efd850186" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "00a49322e3754e149658cbab791e9b00"
I20260806 04:59:39.462023 26796 raft_consensus.cc:3037] T 71c17fc89643456a982ffedda01ed1d2 P f3d52b1be46b435ca76dfde46d3a93b2 [term 0 FOLLOWER]: Advancing to term 1
I20260806 04:59:39.462116 27017 raft_consensus.cc:3037] T 71c17fc89643456a982ffedda01ed1d2 P 00a49322e3754e149658cbab791e9b00 [term 0 FOLLOWER]: Advancing to term 1
I20260806 04:59:39.464154 26796 raft_consensus.cc:2454] T 71c17fc89643456a982ffedda01ed1d2 P f3d52b1be46b435ca76dfde46d3a93b2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f02994845a3c42eea4de4b9efd850186 in term 1.
I20260806 04:59:39.464547 27128 leader_election.cc:302] T 71c17fc89643456a982ffedda01ed1d2 P f02994845a3c42eea4de4b9efd850186 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f02994845a3c42eea4de4b9efd850186, f3d52b1be46b435ca76dfde46d3a93b2; no voters:
I20260806 04:59:39.464807 27346 raft_consensus.cc:2781] T 71c17fc89643456a982ffedda01ed1d2 P f02994845a3c42eea4de4b9efd850186 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 04:59:39.465342 27346 raft_consensus.cc:686] T 71c17fc89643456a982ffedda01ed1d2 P f02994845a3c42eea4de4b9efd850186 [term 1 LEADER]: Becoming Leader. State: Replica: f02994845a3c42eea4de4b9efd850186, State: Running, Role: LEADER
I20260806 04:59:39.466058 27017 raft_consensus.cc:2454] T 71c17fc89643456a982ffedda01ed1d2 P 00a49322e3754e149658cbab791e9b00 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f02994845a3c42eea4de4b9efd850186 in term 1.
I20260806 04:59:39.466193 27346 consensus_queue.cc:237] T 71c17fc89643456a982ffedda01ed1d2 P f02994845a3c42eea4de4b9efd850186 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f02994845a3c42eea4de4b9efd850186" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 38665 } } peers { permanent_uuid: "00a49322e3754e149658cbab791e9b00" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 40351 } } peers { permanent_uuid: "f3d52b1be46b435ca76dfde46d3a93b2" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 38267 } }
I20260806 04:59:39.470750 26531 catalog_manager.cc:5495] T 71c17fc89643456a982ffedda01ed1d2 P f02994845a3c42eea4de4b9efd850186 reported cstate change: term changed from 0 to 1, leader changed from <none> to f02994845a3c42eea4de4b9efd850186 (127.9.96.67). New cstate: current_term: 1 leader_uuid: "f02994845a3c42eea4de4b9efd850186" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f02994845a3c42eea4de4b9efd850186" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 38665 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "00a49322e3754e149658cbab791e9b00" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 40351 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f3d52b1be46b435ca76dfde46d3a93b2" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 38267 } health_report { overall_health: UNKNOWN } } }
I20260806 04:59:39.495131 27327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:59:39.489715 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37058 (local address 127.9.96.67:38665)
0806 04:59:39.489914 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0806 04:59:39.489920 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 04:59:39.490046 (+ 126us) server_negotiation.cc:407] Connection header received
0806 04:59:39.490260 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:59:39.490263 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:59:39.490340 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:59:39.490481 (+ 141us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 04:59:39.490847 (+ 366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:59:39.491622 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:59:39.493042 (+ 1420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:59:39.493437 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:59:39.493598 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:59:39.494123 (+ 525us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 04:59:39.494584 (+ 461us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 04:59:39.494736 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0806 04:59:39.494814 (+ 78us) server_negotiation.cc:299] Negotiation successful
0806 04:59:39.494890 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260806 04:59:39.502511 26796 raft_consensus.cc:1261] T 71c17fc89643456a982ffedda01ed1d2 P f3d52b1be46b435ca76dfde46d3a93b2 [term 1 FOLLOWER]: Refusing update from remote peer f02994845a3c42eea4de4b9efd850186: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 04:59:39.502534 27017 raft_consensus.cc:1261] T 71c17fc89643456a982ffedda01ed1d2 P 00a49322e3754e149658cbab791e9b00 [term 1 FOLLOWER]: Refusing update from remote peer f02994845a3c42eea4de4b9efd850186: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 04:59:39.503543 27350 consensus_queue.cc:1035] T 71c17fc89643456a982ffedda01ed1d2 P f02994845a3c42eea4de4b9efd850186 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3d52b1be46b435ca76dfde46d3a93b2" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 38267 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 04:59:39.503957 27346 consensus_queue.cc:1035] T 71c17fc89643456a982ffedda01ed1d2 P f02994845a3c42eea4de4b9efd850186 [LEADER]: Connected to new peer: Peer: permanent_uuid: "00a49322e3754e149658cbab791e9b00" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 40351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 04:59:39.519701 27357 mvcc.cc:205] Tried to move back new op lower bound from 7315424786433314816 to 7315424786297729024. Current Snapshot: MvccSnapshot[applied={T|T < 7315424786433314816}]
I20260806 04:59:39.532749 26531 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58494:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 04:59:39.533015 26531 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58494:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 04:59:39.536772 26531 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ad7852ba3f9c4377baf400ad6f80588f: Sending DeleteTablet for 3 replicas of tablet 71c17fc89643456a982ffedda01ed1d2
I20260806 04:59:39.537966 27200 tablet_service.cc:1514] Processing DeleteTablet for tablet 71c17fc89643456a982ffedda01ed1d2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 04:59:39 UTC) from {username='kudu', principal='kudu/127.9.96.126@KRBTEST.COM'} at 127.0.0.1:37054
I20260806 04:59:39.537964 26988 tablet_service.cc:1514] Processing DeleteTablet for tablet 71c17fc89643456a982ffedda01ed1d2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 04:59:39 UTC) from {username='kudu', principal='kudu/127.9.96.126@KRBTEST.COM'} at 127.0.0.1:57660
I20260806 04:59:39.538880 27372 tablet_replica.cc:330] T 71c17fc89643456a982ffedda01ed1d2 P f02994845a3c42eea4de4b9efd850186: stopping tablet replica
I20260806 04:59:39.538899 26765 tablet_service.cc:1514] Processing DeleteTablet for tablet 71c17fc89643456a982ffedda01ed1d2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 04:59:39 UTC) from {username='kudu', principal='kudu/127.9.96.126@KRBTEST.COM'} at 127.0.0.1:50030
I20260806 04:59:39.539284 27372 raft_consensus.cc:2229] T 71c17fc89643456a982ffedda01ed1d2 P f02994845a3c42eea4de4b9efd850186 [term 1 LEADER]: Raft consensus shutting down.
I20260806 04:59:39.540030 27372 raft_consensus.cc:2258] T 71c17fc89643456a982ffedda01ed1d2 P f02994845a3c42eea4de4b9efd850186 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 04:59:39.540254 9601 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu pid 26650
I20260806 04:59:39.542028 27372 ts_tablet_manager.cc:1905] T 71c17fc89643456a982ffedda01ed1d2 P f02994845a3c42eea4de4b9efd850186: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 04:59:39.550515 27374 tablet_replica.cc:330] T 71c17fc89643456a982ffedda01ed1d2 P f3d52b1be46b435ca76dfde46d3a93b2: stopping tablet replica
I20260806 04:59:39.550894 27374 raft_consensus.cc:2229] T 71c17fc89643456a982ffedda01ed1d2 P f3d52b1be46b435ca76dfde46d3a93b2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 04:59:39.551606 27372 ts_tablet_manager.cc:1918] T 71c17fc89643456a982ffedda01ed1d2 P f02994845a3c42eea4de4b9efd850186: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 04:59:39.551776 27372 log.cc:1196] T 71c17fc89643456a982ffedda01ed1d2 P f02994845a3c42eea4de4b9efd850186: Deleting WAL directory at /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785992227050689-9601-0/minicluster-data/ts-2/wal/wals/71c17fc89643456a982ffedda01ed1d2
I20260806 04:59:39.552374 27372 ts_tablet_manager.cc:1939] T 71c17fc89643456a982ffedda01ed1d2 P f02994845a3c42eea4de4b9efd850186: Deleting consensus metadata
I20260806 04:59:39.552392 27373 tablet_replica.cc:330] T 71c17fc89643456a982ffedda01ed1d2 P 00a49322e3754e149658cbab791e9b00: stopping tablet replica
I20260806 04:59:39.552762 27373 raft_consensus.cc:2229] T 71c17fc89643456a982ffedda01ed1d2 P 00a49322e3754e149658cbab791e9b00 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 04:59:39.553215 27373 raft_consensus.cc:2258] T 71c17fc89643456a982ffedda01ed1d2 P 00a49322e3754e149658cbab791e9b00 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 04:59:39.553680 27374 raft_consensus.cc:2258] T 71c17fc89643456a982ffedda01ed1d2 P f3d52b1be46b435ca76dfde46d3a93b2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 04:59:39.554034 26509 catalog_manager.cc:4841] TS f02994845a3c42eea4de4b9efd850186 (127.9.96.67:38665): tablet 71c17fc89643456a982ffedda01ed1d2 (table test-table [id=112e77eefa6443baae1a12d00fb9a47e]) successfully deleted
I20260806 04:59:39.556267 27374 ts_tablet_manager.cc:1905] T 71c17fc89643456a982ffedda01ed1d2 P f3d52b1be46b435ca76dfde46d3a93b2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 04:59:39.556584 27373 ts_tablet_manager.cc:1905] T 71c17fc89643456a982ffedda01ed1d2 P 00a49322e3754e149658cbab791e9b00: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 04:59:39.563457 27374 ts_tablet_manager.cc:1918] T 71c17fc89643456a982ffedda01ed1d2 P f3d52b1be46b435ca76dfde46d3a93b2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 04:59:39.563462 27373 ts_tablet_manager.cc:1918] T 71c17fc89643456a982ffedda01ed1d2 P 00a49322e3754e149658cbab791e9b00: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 04:59:39.563618 27374 log.cc:1196] T 71c17fc89643456a982ffedda01ed1d2 P f3d52b1be46b435ca76dfde46d3a93b2: Deleting WAL directory at /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785992227050689-9601-0/minicluster-data/ts-0/wal/wals/71c17fc89643456a982ffedda01ed1d2
I20260806 04:59:39.563721 27373 log.cc:1196] T 71c17fc89643456a982ffedda01ed1d2 P 00a49322e3754e149658cbab791e9b00: Deleting WAL directory at /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785992227050689-9601-0/minicluster-data/ts-1/wal/wals/71c17fc89643456a982ffedda01ed1d2
I20260806 04:59:39.564142 27374 ts_tablet_manager.cc:1939] T 71c17fc89643456a982ffedda01ed1d2 P f3d52b1be46b435ca76dfde46d3a93b2: Deleting consensus metadata
I20260806 04:59:39.564273 27373 ts_tablet_manager.cc:1939] T 71c17fc89643456a982ffedda01ed1d2 P 00a49322e3754e149658cbab791e9b00: Deleting consensus metadata
I20260806 04:59:39.565963 26507 catalog_manager.cc:4841] TS f3d52b1be46b435ca76dfde46d3a93b2 (127.9.96.65:38267): tablet 71c17fc89643456a982ffedda01ed1d2 (table test-table [id=112e77eefa6443baae1a12d00fb9a47e]) successfully deleted
I20260806 04:59:39.566256 26506 catalog_manager.cc:4841] TS 00a49322e3754e149658cbab791e9b00 (127.9.96.66:40351): tablet 71c17fc89643456a982ffedda01ed1d2 (table test-table [id=112e77eefa6443baae1a12d00fb9a47e]) successfully deleted
Aug 06 04:59:39 dist-test-slave-x4qh krb5kdc[25816](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992375, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.96.65@KRBTEST.COM
I20260806 04:59:39.574983 27332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:59:39.543804 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50038 (local address 127.9.96.65:38267)
0806 04:59:39.543993 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0806 04:59:39.543998 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 04:59:39.556956 (+ 12958us) server_negotiation.cc:407] Connection header received
0806 04:59:39.557421 (+ 465us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:59:39.557427 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:59:39.557510 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:59:39.557589 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:59:39.559572 (+ 1983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:59:39.560704 (+ 1132us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:59:39.562744 (+ 2040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:59:39.563495 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:59:39.563586 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:59:39.569282 (+ 5696us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:59:39.569293 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:59:39.569307 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:59:39.569335 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:59:39.572250 (+ 2915us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:59:39.573058 (+ 808us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:59:39.573064 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:59:39.573066 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:59:39.573190 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:59:39.573575 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:59:39.573578 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:59:39.573579 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:59:39.573910 (+ 331us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:59:39.574114 (+ 204us) server_negotiation.cc:1035] Waiting for connection context
0806 04:59:39.574497 (+ 383us) server_negotiation.cc:299] Negotiation successful
0806 04:59:39.574675 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260806 04:59:39.575662 26863 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 04:59:39.678836 9601 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu with pid 26650
I20260806 04:59:39.705219 9601 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu pid 26878
Aug 06 04:59:39 dist-test-slave-x4qh krb5kdc[25816](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992375, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.96.66@KRBTEST.COM
I20260806 04:59:39.721531 27329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:59:39.706206 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57666 (local address 127.9.96.66:40351)
0806 04:59:39.706403 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0806 04:59:39.706409 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 04:59:39.706431 (+ 22us) server_negotiation.cc:407] Connection header received
0806 04:59:39.706502 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:59:39.706514 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:59:39.706601 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:59:39.706710 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:59:39.708209 (+ 1499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:59:39.709042 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:59:39.710497 (+ 1455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:59:39.710900 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:59:39.711041 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:59:39.715464 (+ 4423us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:59:39.715478 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:59:39.715494 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:59:39.715538 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:59:39.718657 (+ 3119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:59:39.719462 (+ 805us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:59:39.719469 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:59:39.719471 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:59:39.719596 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:59:39.719999 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:59:39.720003 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:59:39.720005 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:59:39.720355 (+ 350us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:59:39.720657 (+ 302us) server_negotiation.cc:1035] Waiting for connection context
0806 04:59:39.720967 (+ 310us) server_negotiation.cc:299] Negotiation successful
0806 04:59:39.721191 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260806 04:59:39.722189 27077 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 04:59:39.870781 9601 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu with pid 26878
I20260806 04:59:39.896471 9601 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu pid 27097
Aug 06 04:59:39 dist-test-slave-x4qh krb5kdc[25816](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992375, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.96.67@KRBTEST.COM
I20260806 04:59:39.915681 27327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:59:39.897426 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37060 (local address 127.9.96.67:38665)
0806 04:59:39.897613 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0806 04:59:39.897617 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 04:59:39.897719 (+ 102us) server_negotiation.cc:407] Connection header received
0806 04:59:39.897925 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:59:39.897929 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:59:39.897989 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:59:39.898071 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:59:39.899972 (+ 1901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:59:39.900773 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:59:39.902850 (+ 2077us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:59:39.903314 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:59:39.903513 (+ 199us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:59:39.909262 (+ 5749us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:59:39.909278 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:59:39.909294 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:59:39.909332 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:59:39.912591 (+ 3259us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:59:39.913312 (+ 721us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:59:39.913327 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:59:39.913330 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:59:39.913528 (+ 198us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:59:39.913919 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:59:39.913924 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:59:39.913928 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:59:39.914312 (+ 384us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:59:39.914635 (+ 323us) server_negotiation.cc:1035] Waiting for connection context
0806 04:59:39.914961 (+ 326us) server_negotiation.cc:299] Negotiation successful
0806 04:59:39.915205 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260806 04:59:39.916469 27296 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 04:59:40.039950 9601 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu with pid 27097
I20260806 04:59:40.066802 9601 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu pid 26483
I20260806 04:59:40.068004 26572 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 04:59:40.179174 9601 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu with pid 26483
I20260806 04:59:40.212035 9601 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260806 04:59:40.557734 9601 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260806 04:59:40.558068 9601 mini_ranger.cc:66] Stopping Ranger...
I20260806 04:59:40.903894 9601 mini_ranger.cc:68] Stopped Ranger
2026-08-06 04:59:40.904 UTC [25858] LOG: received smart shutdown request
2026-08-06 04:59:40.907 UTC [25858] LOG: background worker "logical replication launcher" (PID 25871) exited with exit code 1
2026-08-06 04:59:40.908 UTC [25862] LOG: shutting down
2026-08-06 04:59:41.054 UTC [25858] LOG: database system is shut down
2026-08-06T04:59:41Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (50576 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785992227050689-9601-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 04:59:41 dist-test-slave-x4qh krb5kdc[27396](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 04:59:41 dist-test-slave-x4qh krb5kdc[27396](info): set up 2 sockets
Aug 06 04:59:41 dist-test-slave-x4qh krb5kdc[27396](info): commencing operation
krb5kdc: starting...
W20260806 04:59:42.500986 9601 mini_kdc.cc:121] Time spent starting KDC: real 1.271s user 0.000s sys 0.032s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785992227050689-9601-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785992227050689-9601-0/krb5kdc/test-user.keytab.
Aug 06 04:59:42 dist-test-slave-x4qh krb5kdc[27396](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992382, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T04:59:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T04:59:42Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.9.96.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.9.96.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.9.96.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785992227050689-9601-0/krb5kdc/rangeradmin_127.9.96.84@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.9.96.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785992227050689-9601-0/krb5kdc/rangeradmin_127.9.96.84@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.9.96.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.9.96.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.9.96.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785992227050689-9601-0/krb5kdc/rangerlookup_127.9.96.84@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.9.96.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785992227050689-9601-0/krb5kdc/rangerlookup_127.9.96.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.9.96.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.9.96.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.9.96.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785992227050689-9601-0/krb5kdc/HTTP_127.9.96.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.9.96.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785992227050689-9601-0/krb5kdc/HTTP_127.9.96.84@KRBTEST.COM.keytab.
I20260806 04:59:42.844015 9601 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785992227050689-9601-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskOfb9uZ/build/asan/bin/postgres/pg_ctl -D /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785992227050689-9601-0/minicluster-data/postgres -l logfile start
2026-08-06 04:59:45.223 UTC [27448] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-06 04:59:45.223 UTC [27448] LOG: listening on IPv4 address "127.9.96.84", port 46381
2026-08-06 04:59:45.227 UTC [27448] LOG: listening on Unix socket "/tmp/.s.PGSQL.46381"
2026-08-06 04:59:45.249 UTC [27454] LOG: database system was shut down at 2026-08-06 04:59:43 UTC
2026-08-06 04:59:45.253 UTC [27448] LOG: database system is ready to accept connections
I20260806 04:59:46.403443 9601 mini_postgres.cc:96] Postgres bound to 46381
2026-08-06 04:59:46.425 UTC [27472] FATAL: database "slave" does not exist
127.9.96.84:46381 - accepting connections
I20260806 04:59:46.426035 9601 mini_ranger.cc:161] Starting Ranger...
I20260806 04:59:46.453938 9601 mini_ranger.cc:84] Created miniranger Postgres user
I20260806 04:59:46.922700 9601 mini_ranger.cc:87] Created ranger Postgres database
I20260806 04:59:46.922829 9601 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785992227050689-9601-0/minicluster-data/ranger-admin
2026-08-06 04:59:47,205 [I] DB FLAVOR :POSTGRES
2026-08-06 04:59:47,206 [I] --------- Verifying Ranger DB connection ---------
2026-08-06 04:59:47,206 [I] Checking connection..
2026-08-06 04:59:47,206 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskOfb9uZ/build/asan/bin/postgresql.jar:/tmp/dist-test-taskOfb9uZ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.96.84:46381/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-06 04:59:47,473 [I] Checking connection passed.
2026-08-06 04:59:47,473 [I] --------- Verifying version history table ---------
2026-08-06 04:59:47,473 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskOfb9uZ/build/asan/bin/postgresql.jar:/tmp/dist-test-taskOfb9uZ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.96.84:46381/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-06 04:59:47,735 [I] Table x_db_version_h does not exist in database ranger
2026-08-06 04:59:47,736 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskOfb9uZ/build/asan/bin/postgresql.jar:/tmp/dist-test-taskOfb9uZ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.96.84:46381/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-06 04:59:48,003 [I] Table x_db_version_h does not exist in database ranger
2026-08-06 04:59:48,003 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-06 04:59:48,004 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskOfb9uZ/build/asan/bin/postgresql.jar:/tmp/dist-test-taskOfb9uZ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.96.84:46381/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskOfb9uZ/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-06 04:59:48.233 UTC [27565] WARNING: there is no transaction in progress
2026-08-06 04:59:48,241 [I] create_dbversion_catalog.sql file imported successfully
2026-08-06 04:59:48,242 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskOfb9uZ/build/asan/bin/postgresql.jar:/tmp/dist-test-taskOfb9uZ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.96.84:46381/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-06 04:59:48,503 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-06 04:59:48,503 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-06 04:59:48,503 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskOfb9uZ/build/asan/bin/postgresql.jar:/tmp/dist-test-taskOfb9uZ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.96.84:46381/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-06 04:59:48,766 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskOfb9uZ/build/asan/bin/postgresql.jar:/tmp/dist-test-taskOfb9uZ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.96.84:46381/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-06 04:59:49,018 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskOfb9uZ/build/asan/bin/postgresql.jar:/tmp/dist-test-taskOfb9uZ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.96.84:46381/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-x4qh.c.gcp-upstream.internal','N') ;"
2026-08-06 04:59:49,259 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskOfb9uZ/build/asan/bin/postgresql.jar:/tmp/dist-test-taskOfb9uZ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.96.84:46381/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-06 04:59:49,494 [I] Table x_portal_user does not exist in database ranger
2026-08-06 04:59:49,494 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskOfb9uZ/build/asan/bin/postgresql.jar:/tmp/dist-test-taskOfb9uZ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.96.84:46381/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-06 04:59:49,742 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-06 04:59:49,743 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskOfb9uZ/build/asan/bin/postgresql.jar:/tmp/dist-test-taskOfb9uZ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.96.84:46381/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-06 04:59:49,997 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskOfb9uZ/build/asan/bin/postgresql.jar:/tmp/dist-test-taskOfb9uZ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.96.84:46381/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-06 04:59:50,257 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-06 04:59:50,257 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskOfb9uZ/build/asan/bin/postgresql.jar:/tmp/dist-test-taskOfb9uZ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.96.84:46381/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskOfb9uZ/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-06 04:59:51.188 UTC [27728] WARNING: there is no transaction in progress
2026-08-06 04:59:51.203 UTC [27728] WARNING: there is no transaction in progress
2026-08-06 04:59:51.218 UTC [27728] WARNING: there is no transaction in progress
2026-08-06 04:59:51.232 UTC [27728] WARNING: there is no transaction in progress
2026-08-06 04:59:51.246 UTC [27728] WARNING: there is no transaction in progress
2026-08-06 04:59:51.263 UTC [27728] WARNING: there is no transaction in progress
2026-08-06 04:59:51.314 UTC [27728] WARNING: there is no transaction in progress
2026-08-06 04:59:51.321 UTC [27728] WARNING: there is no transaction in progress
2026-08-06 04:59:51.334 UTC [27728] WARNING: there is no transaction in progress
2026-08-06 04:59:51.343 UTC [27728] WARNING: there is no transaction in progress
2026-08-06 04:59:51.352 UTC [27728] WARNING: there is no transaction in progress
2026-08-06 04:59:51.365 UTC [27728] WARNING: there is no transaction in progress
2026-08-06 04:59:51.374 UTC [27728] WARNING: there is no transaction in progress
2026-08-06 04:59:51.380 UTC [27728] WARNING: there is no transaction in progress
2026-08-06 04:59:51.386 UTC [27728] WARNING: there is no transaction in progress
2026-08-06 04:59:51.392 UTC [27728] WARNING: there is no transaction in progress
2026-08-06 04:59:51,981 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-06 04:59:51,981 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskOfb9uZ/build/asan/bin/postgresql.jar:/tmp/dist-test-taskOfb9uZ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.96.84:46381/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-06 04:59:52,228 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-06 04:59:52,229 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskOfb9uZ/build/asan/bin/postgresql.jar:/tmp/dist-test-taskOfb9uZ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.96.84:46381/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-06 04:59:52,461 [I] Table x_portal_user already exists in database 'ranger'
2026-08-06 04:59:52,461 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskOfb9uZ/build/asan/bin/postgresql.jar:/tmp/dist-test-taskOfb9uZ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.96.84:46381/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-06 04:59:52,724 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-06 04:59:52,725 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskOfb9uZ/build/asan/bin/postgresql.jar:/tmp/dist-test-taskOfb9uZ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.96.84:46381/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-06 04:59:52,973 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskOfb9uZ/build/asan/bin/postgresql.jar:/tmp/dist-test-taskOfb9uZ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.96.84:46381/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-06 04:59:53,236 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskOfb9uZ/build/asan/bin/postgresql.jar:/tmp/dist-test-taskOfb9uZ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.96.84:46381/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-x4qh.c.gcp-upstream.internal';"
2026-08-06 04:59:53,494 [I] CORE_DB_SCHEMA import status has been updated
2026-08-06 04:59:53,494 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskOfb9uZ/build/asan/bin/postgresql.jar:/tmp/dist-test-taskOfb9uZ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.96.84:46381/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-06 04:59:53,758 [I] DB_PATCHES have already been applied
I20260806 04:59:53.764514 9601 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785992227050689-9601-0/minicluster-data/ranger-admin:/tmp/dist-test-taskOfb9uZ/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskOfb9uZ/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskOfb9uZ/build/asan/bin/postgresql.jar:/tmp/dist-test-taskOfb9uZ/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260806 04:59:53.764626 9601 mini_ranger.cc:193] Using host: 127.9.96.84
I20260806 04:59:53.783624 9601 mini_ranger.cc:227] Ranger admin URL: http://127.9.96.84:44613
Aug 06, 2026 4:59:54 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 06, 2026 4:59:54 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 06, 2026 4:59:54 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785992227050689-9601-0/minicluster-data/ranger-admin/ews/webapp
Aug 06, 2026 4:59:54 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785992227050689-9601-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 06, 2026 4:59:54 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785992227050689-9601-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 06, 2026 4:59:54 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785992227050689-9601-0/minicluster-data/ranger-admin/ews/webapp].
Aug 06, 2026 4:59:54 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260806 04:59:55.174824 9601 mini_ranger.cc:160] Time spent starting Ranger: real 8.749s user 0.005s sys 0.047s
Aug 06, 2026 5:00:15 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 06, 2026 5:00:15 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.UserREST
Aug 06, 2026 5:00:15 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 06, 2026 5:00:15 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 06, 2026 5:00:15 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 06, 2026 5:00:15 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 06, 2026 5:00:15 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 06, 2026 5:00:15 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 06, 2026 5:00:15 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 06, 2026 5:00:15 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 06, 2026 5:00:15 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 06, 2026 5:00:15 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 06, 2026 5:00:15 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 06, 2026 5:00:15 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 06, 2026 5:00:15 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 06, 2026 5:00:15 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 06, 2026 5:00:15 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 06, 2026 5:00:16 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260806 05:00:17.236961 9601 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.9.96.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.9.96.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.9.96.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785992227050689-9601-0/krb5kdc/rangerkms_127.9.96.84@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.9.96.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785992227050689-9601-0/krb5kdc/rangerkms_127.9.96.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.9.96.84@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.9.96.84@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.9.96.84@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785992227050689-9601-0/krb5kdc/HTTP_127.9.96.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.9.96.84@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785992227050689-9601-0/krb5kdc/HTTP_127.9.96.84@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 06 05:00:17 dist-test-slave-x4qh krb5kdc[27396](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992417, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260806 05:00:17.455915 9601 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260806 05:00:17.482275 9601 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260806 05:00:18.334287 9601 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260806 05:00:18.334421 9601 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785992227050689-9601-0/minicluster-data/ranger-kms
I20260806 05:00:18.334477 9601 mini_ranger_kms.cc:227] Using postgres at 127.9.96.84:46381
2026-08-06 05:00:18,695 [I] DB FLAVOR :POSTGRES
2026-08-06 05:00:18,696 [I] --------- Verifying Ranger DB connection ---------
2026-08-06 05:00:18,696 [I] Checking connection
2026-08-06 05:00:18,696 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskOfb9uZ/build/asan/bin/postgresql.jar:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785992227050689-9601-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.96.84:46381/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-06 05:00:18,941 [I] connection success
2026-08-06 05:00:18,942 [I] --------- Verifying Ranger DB tables ---------
2026-08-06 05:00:18,942 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-06 05:00:18,942 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskOfb9uZ/build/asan/bin/postgresql.jar:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785992227050689-9601-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.96.84:46381/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-06 05:00:19,180 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-06 05:00:19,181 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-06 05:00:19,181 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-06 05:00:19,181 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskOfb9uZ/build/asan/bin/postgresql.jar:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785992227050689-9601-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.96.84:46381/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785992227050689-9601-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-06 05:00:19,473 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260806 05:00:19.945307 9601 mini_ranger_kms.cc:310] Created kms service
I20260806 05:00:20.145491 9601 mini_ranger_kms.cc:326] Created kudu user
I20260806 05:00:20.190552 9601 mini_ranger_kms.cc:343] Created rangerkms user
Aug 06 05:00:20 dist-test-slave-x4qh krb5kdc[27396](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992417, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.9.96.84@KRBTEST.COM
I20260806 05:00:20.566776 9601 mini_ranger_kms.cc:384] Added ranger policy
I20260806 05:00:20.567126 9601 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785992227050689-9601-0/minicluster-data/ranger-kms:/tmp/dist-test-taskOfb9uZ/build/asan/bin/postgresql.jar:/tmp/dist-test-taskOfb9uZ/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskOfb9uZ/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260806 05:00:20.567183 9601 mini_ranger_kms.cc:240] Using host: 127.9.96.84
I20260806 05:00:20.589605 9601 mini_ranger_kms.cc:276] Ranger KMS PID: 28020
I20260806 05:00:20.589764 9601 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.9.96.84:43097
Aug 06, 2026 5:00:21 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 06, 2026 5:00:21 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 06, 2026 5:00:21 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785992227050689-9601-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 06, 2026 5:00:21 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785992227050689-9601-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 06, 2026 5:00:22 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785992227050689-9601-0/minicluster-data/ranger-kms/ews/webapp].
Aug 06, 2026 5:00:22 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 06 05:00:22 dist-test-slave-x4qh krb5kdc[27396](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1785992422, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.9.96.84@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 05:00:23.391110 9601 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.935s user 0.051s sys 0.342s
I20260806 05:00:23.391403 9601 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260806 05:00:23.391592 9601 mini_ranger_kms.cc:401] 127.9.96.84:43097/kms/v1/keys
Aug 06 05:00:23 dist-test-slave-x4qh krb5kdc[27396](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992422, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.9.96.84@KRBTEST.COM for HTTP/127.9.96.84@KRBTEST.COM
Aug 06 05:00:23 dist-test-slave-x4qh krb5kdc[27396](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992422, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.9.96.84@KRBTEST.COM for HTTP/127.9.96.84@KRBTEST.COM
Aug 06 05:00:26 dist-test-slave-x4qh krb5kdc[27396](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992426, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.9.96.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.96.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.96.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.126.keytab.
Entry for principal kudu/127.9.96.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.126.keytab.
I20260806 05:00:26.078734 9601 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
/tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785992227050689-9601-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785992227050689-9601-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785992227050689-9601-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785992227050689-9601-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.96.126:45703
--webserver_interface=127.9.96.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.126.keytab
--principal=kudu/127.9.96.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.96.84:35379
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.96.126:45703
--ranger_config_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785992227050689-9601-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.9.96.84:43097/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785992227050689-9601-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785992227050689-9601-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785992227050689-9601-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 05:00:26.410308 28094 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 05:00:26.410614 28094 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 05:00:26.410712 28094 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 05:00:26.417954 28094 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 05:00:26.418069 28094 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 05:00:26.418118 28094 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 05:00:26.418176 28094 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 05:00:26.418229 28094 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 05:00:26.428174 28094 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.96.84:35379
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785992227050689-9601-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785992227050689-9601-0/minicluster-data/master-0/wal
--ranger_kms_url=127.9.96.84:43097/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.9.96.126:45703
--ranger_config_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785992227050689-9601-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.126.keytab
--principal=kudu/127.9.96.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.96.126:45703
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785992227050689-9601-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.96.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785992227050689-9601-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 03:56:12 UTC on ccycloud.quasar-guijbm.root.comops.site
build id 4569618
ASAN enabled
I20260806 05:00:26.429495 28094 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 05:00:26.430843 28094 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 05:00:26.443176 28103 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 05:00:26.443143 28104 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 05:00:26.443090 28106 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 05:00:26.443758 28094 server_base.cc:1034] running on GCE node
I20260806 05:00:26.444583 28094 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 05:00:26.445930 28094 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 05:00:26.447135 28094 hybrid_clock.cc:648] HybridClock initialized: now 1785992426447103 us; error 29 us; skew 500 ppm
I20260806 05:00:26.447531 28094 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 05:00:26 dist-test-slave-x4qh krb5kdc[27396](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992426, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.96.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 05:00:26.454207 28094 init.cc:377] Logged in from keytab as kudu/127.9.96.126@KRBTEST.COM (short username kudu)
I20260806 05:00:26.455686 28094 webserver.cc:460] Webserver started at http://127.9.96.126:34245/ using document root <none> and password file <none>
I20260806 05:00:26.456357 28094 fs_manager.cc:359] Metadata directory not provided
I20260806 05:00:26.456493 28094 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 05:00:26.456822 28094 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 05:00:26 dist-test-slave-x4qh krb5kdc[27396](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992426, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.96.126@KRBTEST.COM for HTTP/127.9.96.84@KRBTEST.COM
I20260806 05:00:26.513897 28094 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785992227050689-9601-0/minicluster-data/master-0/data/instance:
uuid: "57daa83eda8242b5af95a57caa6b61a8"
format_stamp: "Formatted at 2026-08-06 05:00:26 on dist-test-slave-x4qh"
server_key: "f6c9746c3871c953a32a97a1f4c7c890"
server_key_iv: "569c07ed3d500e6d3e9b613d842af8d8"
server_key_version: "kuduclusterkey@0"
I20260806 05:00:26.514838 28094 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785992227050689-9601-0/minicluster-data/master-0/wal/instance:
uuid: "57daa83eda8242b5af95a57caa6b61a8"
format_stamp: "Formatted at 2026-08-06 05:00:26 on dist-test-slave-x4qh"
server_key: "f6c9746c3871c953a32a97a1f4c7c890"
server_key_iv: "569c07ed3d500e6d3e9b613d842af8d8"
server_key_version: "kuduclusterkey@0"
I20260806 05:00:26.520537 28094 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.002s
I20260806 05:00:26.540555 28119 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 05:00:26.542110 28094 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260806 05:00:26.542316 28094 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785992227050689-9601-0/minicluster-data/master-0/data,/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785992227050689-9601-0/minicluster-data/master-0/wal
uuid: "57daa83eda8242b5af95a57caa6b61a8"
format_stamp: "Formatted at 2026-08-06 05:00:26 on dist-test-slave-x4qh"
server_key: "f6c9746c3871c953a32a97a1f4c7c890"
server_key_iv: "569c07ed3d500e6d3e9b613d842af8d8"
server_key_version: "kuduclusterkey@0"
I20260806 05:00:26.542544 28094 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785992227050689-9601-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785992227050689-9601-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785992227050689-9601-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 05:00:26.579010 28094 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 05:00:26.584139 28094 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 05:00:26.584568 28094 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 05:00:26.610750 28094 rpc_server.cc:307] RPC server started. Bound to: 127.9.96.126:45703
I20260806 05:00:26.610844 28197 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.96.126:45703 every 8 connection(s)
I20260806 05:00:26.612270 28094 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785992227050689-9601-0/minicluster-data/master-0/data/info.pb
I20260806 05:00:26.617204 28198 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 05:00:26.621773 9601 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu as pid 28094
I20260806 05:00:26.622051 9601 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785992227050689-9601-0/minicluster-data/master-0/wal/instance
I20260806 05:00:26.622989 9601 external_mini_cluster.cc:1428] Setting key dce35e46125be3798900bd8bdeede2ba
I20260806 05:00:26.634068 28198 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 57daa83eda8242b5af95a57caa6b61a8: Bootstrap starting.
Aug 06 05:00:26 dist-test-slave-x4qh krb5kdc[27396](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992426, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.96.126@KRBTEST.COM
I20260806 05:00:26.638440 28198 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 57daa83eda8242b5af95a57caa6b61a8: Neither blocks nor log segments found. Creating new log.
I20260806 05:00:26.639889 28198 log.cc:824] T 00000000000000000000000000000000 P 57daa83eda8242b5af95a57caa6b61a8: Log is configured to *not* fsync() on all Append() calls
I20260806 05:00:26.643961 28198 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 57daa83eda8242b5af95a57caa6b61a8: No bootstrap required, opened a new log
I20260806 05:00:26.648634 28201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:00:26.625636 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46392 (local address 127.9.96.126:45703)
0806 05:00:26.626470 (+ 834us) server_negotiation.cc:206] Beginning negotiation
0806 05:00:26.626488 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0806 05:00:26.626544 (+ 56us) server_negotiation.cc:407] Connection header received
0806 05:00:26.628616 (+ 2072us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 05:00:26.628638 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 05:00:26.629008 (+ 370us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 05:00:26.629489 (+ 481us) server_negotiation.cc:226] Negotiated authn=SASL
0806 05:00:26.630865 (+ 1376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:00:26.632257 (+ 1392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:00:26.633602 (+ 1345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:00:26.634342 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:00:26.634555 (+ 213us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:00:26.638894 (+ 4339us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 05:00:26.638902 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 05:00:26.638920 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 05:00:26.638965 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0806 05:00:26.642933 (+ 3968us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 05:00:26.643962 (+ 1029us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 05:00:26.643973 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 05:00:26.643984 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0806 05:00:26.644187 (+ 203us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 05:00:26.644739 (+ 552us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 05:00:26.644747 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 05:00:26.644756 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0806 05:00:26.645559 (+ 803us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 05:00:26.645928 (+ 369us) server_negotiation.cc:1035] Waiting for connection context
0806 05:00:26.646460 (+ 532us) server_negotiation.cc:299] Negotiation successful
0806 05:00:26.646941 (+ 481us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":595,"thread_start_us":392,"threads_started":1}
I20260806 05:00:26.653836 28198 raft_consensus.cc:348] T 00000000000000000000000000000000 P 57daa83eda8242b5af95a57caa6b61a8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57daa83eda8242b5af95a57caa6b61a8" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 45703 } }
I20260806 05:00:26.654208 28198 raft_consensus.cc:374] T 00000000000000000000000000000000 P 57daa83eda8242b5af95a57caa6b61a8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 05:00:26.654328 28198 raft_consensus.cc:729] T 00000000000000000000000000000000 P 57daa83eda8242b5af95a57caa6b61a8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57daa83eda8242b5af95a57caa6b61a8, State: Initialized, Role: FOLLOWER
I20260806 05:00:26.655009 28198 consensus_queue.cc:260] T 00000000000000000000000000000000 P 57daa83eda8242b5af95a57caa6b61a8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57daa83eda8242b5af95a57caa6b61a8" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 45703 } }
I20260806 05:00:26.655247 28198 raft_consensus.cc:388] T 00000000000000000000000000000000 P 57daa83eda8242b5af95a57caa6b61a8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 05:00:26.655411 28198 raft_consensus.cc:482] T 00000000000000000000000000000000 P 57daa83eda8242b5af95a57caa6b61a8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 05:00:26.655623 28198 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 57daa83eda8242b5af95a57caa6b61a8 [term 0 FOLLOWER]: Advancing to term 1
I20260806 05:00:26.658233 28198 raft_consensus.cc:504] T 00000000000000000000000000000000 P 57daa83eda8242b5af95a57caa6b61a8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57daa83eda8242b5af95a57caa6b61a8" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 45703 } }
I20260806 05:00:26.658778 28198 leader_election.cc:302] T 00000000000000000000000000000000 P 57daa83eda8242b5af95a57caa6b61a8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 57daa83eda8242b5af95a57caa6b61a8; no voters:
I20260806 05:00:26.659453 28198 leader_election.cc:288] T 00000000000000000000000000000000 P 57daa83eda8242b5af95a57caa6b61a8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 05:00:26.659560 28203 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 57daa83eda8242b5af95a57caa6b61a8 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 05:00:26.660996 28203 raft_consensus.cc:686] T 00000000000000000000000000000000 P 57daa83eda8242b5af95a57caa6b61a8 [term 1 LEADER]: Becoming Leader. State: Replica: 57daa83eda8242b5af95a57caa6b61a8, State: Running, Role: LEADER
I20260806 05:00:26.661715 28198 sys_catalog.cc:564] T 00000000000000000000000000000000 P 57daa83eda8242b5af95a57caa6b61a8 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 05:00:26.661602 28203 consensus_queue.cc:237] T 00000000000000000000000000000000 P 57daa83eda8242b5af95a57caa6b61a8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57daa83eda8242b5af95a57caa6b61a8" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 45703 } }
I20260806 05:00:26.677610 28204 sys_catalog.cc:455] T 00000000000000000000000000000000 P 57daa83eda8242b5af95a57caa6b61a8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "57daa83eda8242b5af95a57caa6b61a8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57daa83eda8242b5af95a57caa6b61a8" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 45703 } } }
I20260806 05:00:26.678267 28204 sys_catalog.cc:458] T 00000000000000000000000000000000 P 57daa83eda8242b5af95a57caa6b61a8 [sys.catalog]: This master's current role is: LEADER
I20260806 05:00:26.676935 28205 sys_catalog.cc:455] T 00000000000000000000000000000000 P 57daa83eda8242b5af95a57caa6b61a8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 57daa83eda8242b5af95a57caa6b61a8. Latest consensus state: current_term: 1 leader_uuid: "57daa83eda8242b5af95a57caa6b61a8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57daa83eda8242b5af95a57caa6b61a8" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 45703 } } }
I20260806 05:00:26.679523 28205 sys_catalog.cc:458] T 00000000000000000000000000000000 P 57daa83eda8242b5af95a57caa6b61a8 [sys.catalog]: This master's current role is: LEADER
I20260806 05:00:26.680733 28218 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 05:00:26.685873 28218 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 05:00:26.686715 28198 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785992227050689-9601-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260806 05:00:26.723846 28218 catalog_manager.cc:1347] Generated new cluster ID: 5a864725429f4e8082b1687f5beb9217
I20260806 05:00:26.724004 28218 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 05:00:26.756381 28218 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 05:00:26.758843 28218 catalog_manager.cc:1504] Loading token signing keys...
I20260806 05:00:26.772773 28218 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 57daa83eda8242b5af95a57caa6b61a8: Generated new TSK 0
I20260806 05:00:26.775661 28218 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.9.96.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.96.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.96.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.65.keytab.
Entry for principal kudu/127.9.96.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.65.keytab.
I20260806 05:00:27.593540 9601 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
/tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785992227050689-9601-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785992227050689-9601-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785992227050689-9601-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785992227050689-9601-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.9.96.84:43097/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.96.65:0
--local_ip_for_outbound_sockets=127.9.96.65
--webserver_interface=127.9.96.65
--webserver_port=0
--tserver_master_addrs=127.9.96.126:45703
--keytab_file=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.65.keytab
--principal=kudu/127.9.96.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.96.84:35379
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785992227050689-9601-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785992227050689-9601-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785992227050689-9601-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 05:00:27 dist-test-slave-x4qh krb5kdc[27396](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1785992427, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.96.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260806 05:00:28.046527 28262 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 05:00:28.046819 28262 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 05:00:28.047042 28262 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 05:00:28.054615 28262 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 05:00:28.054781 28262 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 05:00:28.054989 28262 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.96.65
I20260806 05:00:28.066493 28262 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.96.84:35379
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785992227050689-9601-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785992227050689-9601-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.9.96.84:43097/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.65.keytab
--principal=kudu/127.9.96.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.96.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785992227050689-9601-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.96.65
--webserver_port=0
--tserver_master_addrs=127.9.96.126:45703
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.96.65
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785992227050689-9601-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 03:56:12 UTC on ccycloud.quasar-guijbm.root.comops.site
build id 4569618
ASAN enabled
I20260806 05:00:28.067850 28262 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 05:00:28.069263 28262 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 05:00:28.092825 28274 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 05:00:28.094620 28275 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 05:00:28.100472 28277 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 05:00:28.101742 28262 server_base.cc:1034] running on GCE node
I20260806 05:00:28.102456 28262 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 05:00:28.103534 28262 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 05:00:28.104776 28262 hybrid_clock.cc:648] HybridClock initialized: now 1785992428104705 us; error 74 us; skew 500 ppm
I20260806 05:00:28.105183 28262 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 05:00:28 dist-test-slave-x4qh krb5kdc[27396](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992428, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.96.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 05:00:28.108660 28262 init.cc:377] Logged in from keytab as kudu/127.9.96.65@KRBTEST.COM (short username kudu)
I20260806 05:00:28.111073 28262 webserver.cc:460] Webserver started at http://127.9.96.65:44839/ using document root <none> and password file <none>
I20260806 05:00:28.111979 28262 fs_manager.cc:359] Metadata directory not provided
I20260806 05:00:28.112129 28262 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 05:00:28.112481 28262 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 05:00:28 dist-test-slave-x4qh krb5kdc[27396](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992428, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.96.65@KRBTEST.COM for HTTP/127.9.96.84@KRBTEST.COM
I20260806 05:00:28.130787 28262 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785992227050689-9601-0/minicluster-data/ts-0/data/instance:
uuid: "871aee67d4564a37b4290d09c6257fc2"
format_stamp: "Formatted at 2026-08-06 05:00:28 on dist-test-slave-x4qh"
server_key: "43f2b9b9fef214cc19369c5b848fc940"
server_key_iv: "de9ff71b8a3b4360feb82be6355b846c"
server_key_version: "kuduclusterkey@0"
I20260806 05:00:28.132086 28262 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785992227050689-9601-0/minicluster-data/ts-0/wal/instance:
uuid: "871aee67d4564a37b4290d09c6257fc2"
format_stamp: "Formatted at 2026-08-06 05:00:28 on dist-test-slave-x4qh"
server_key: "43f2b9b9fef214cc19369c5b848fc940"
server_key_iv: "de9ff71b8a3b4360feb82be6355b846c"
server_key_version: "kuduclusterkey@0"
I20260806 05:00:28.138458 28262 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.001s
I20260806 05:00:28.178987 28287 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 05:00:28.181245 28262 fs_manager.cc:725] Time spent opening block manager: real 0.017s user 0.007s sys 0.008s
I20260806 05:00:28.181483 28262 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785992227050689-9601-0/minicluster-data/ts-0/data,/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785992227050689-9601-0/minicluster-data/ts-0/wal
uuid: "871aee67d4564a37b4290d09c6257fc2"
format_stamp: "Formatted at 2026-08-06 05:00:28 on dist-test-slave-x4qh"
server_key: "43f2b9b9fef214cc19369c5b848fc940"
server_key_iv: "de9ff71b8a3b4360feb82be6355b846c"
server_key_version: "kuduclusterkey@0"
I20260806 05:00:28.181778 28262 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785992227050689-9601-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785992227050689-9601-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785992227050689-9601-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 05:00:28.199888 28262 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 05:00:28.204191 28262 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 05:00:28.204546 28262 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 05:00:28.205637 28262 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 05:00:28.207182 28262 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 05:00:28.207289 28262 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 05:00:28.207472 28262 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 05:00:28.207552 28262 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 05:00:28.283703 28262 rpc_server.cc:307] RPC server started. Bound to: 127.9.96.65:44843
I20260806 05:00:28.283726 28462 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.96.65:44843 every 8 connection(s)
I20260806 05:00:28.285341 28262 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785992227050689-9601-0/minicluster-data/ts-0/data/info.pb
I20260806 05:00:28.295975 9601 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu as pid 28262
I20260806 05:00:28.296249 9601 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785992227050689-9601-0/minicluster-data/ts-0/wal/instance
I20260806 05:00:28.297300 9601 external_mini_cluster.cc:1428] Setting key 69d89393d4d83ee6331cb671aea5e36a
Aug 06 05:00:28 dist-test-slave-x4qh krb5kdc[27396](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992428, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.96.65@KRBTEST.COM for kudu/127.9.96.126@KRBTEST.COM
I20260806 05:00:28.326856 28466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:00:28.289488 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:45703 (local address 127.9.96.65:60581)
0806 05:00:28.300716 (+ 11228us) negotiation.cc:107] Waiting for socket to connect
0806 05:00:28.300790 (+ 74us) client_negotiation.cc:174] Beginning negotiation
0806 05:00:28.303336 (+ 2546us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 05:00:28.305817 (+ 2481us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 05:00:28.305830 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 05:00:28.306496 (+ 666us) client_negotiation.cc:189] Negotiated authn=SASL
0806 05:00:28.307393 (+ 897us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 05:00:28.307431 (+ 38us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 05:00:28.309004 (+ 1573us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 05:00:28.309011 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 05:00:28.311134 (+ 2123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 05:00:28.311161 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 05:00:28.312221 (+ 1060us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 05:00:28.312229 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 05:00:28.312529 (+ 300us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:00:28.313336 (+ 807us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 05:00:28.313379 (+ 43us) client_negotiation.cc:656] calling sasl_client_start()
0806 05:00:28.316194 (+ 2815us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 05:00:28.321559 (+ 5365us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 05:00:28.321571 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 05:00:28.321592 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0806 05:00:28.322261 (+ 669us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 05:00:28.322801 (+ 540us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 05:00:28.322809 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 05:00:28.322816 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0806 05:00:28.323122 (+ 306us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 05:00:28.323911 (+ 789us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 05:00:28.323921 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 05:00:28.324407 (+ 486us) client_negotiation.cc:769] Sending connection context
0806 05:00:28.324713 (+ 306us) client_negotiation.cc:240] Negotiation successful
0806 05:00:28.325341 (+ 628us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10878,"thread_start_us":13791,"threads_started":1}
I20260806 05:00:28.328642 28465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:00:28.288993 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.96.65:60581 (local address 127.9.96.126:45703)
0806 05:00:28.304634 (+ 15641us) server_negotiation.cc:206] Beginning negotiation
0806 05:00:28.304643 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 05:00:28.304670 (+ 27us) server_negotiation.cc:407] Connection header received
0806 05:00:28.305135 (+ 465us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 05:00:28.305141 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 05:00:28.305237 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 05:00:28.305347 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0806 05:00:28.307620 (+ 2273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:00:28.308799 (+ 1179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:00:28.311296 (+ 2497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:00:28.311984 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:00:28.312119 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:00:28.316469 (+ 4350us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 05:00:28.316476 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 05:00:28.316481 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 05:00:28.316523 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0806 05:00:28.321371 (+ 4848us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 05:00:28.322442 (+ 1071us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 05:00:28.322450 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 05:00:28.322456 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 05:00:28.322649 (+ 193us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 05:00:28.323279 (+ 630us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 05:00:28.323286 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 05:00:28.323293 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 05:00:28.323728 (+ 435us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 05:00:28.327725 (+ 3997us) server_negotiation.cc:1035] Waiting for connection context
0806 05:00:28.327891 (+ 166us) server_negotiation.cc:299] Negotiation successful
0806 05:00:28.328154 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15504,"thread_start_us":17757,"threads_started":1}
I20260806 05:00:28.330358 28463 heartbeater.cc:347] Connected to a master server at 127.9.96.126:45703
I20260806 05:00:28.330705 28463 heartbeater.cc:464] Registering TS with master...
I20260806 05:00:28.331693 28463 heartbeater.cc:511] Master 127.9.96.126:45703 requested a full tablet report, sending...
I20260806 05:00:28.334422 28147 ts_manager.cc:194] Registered new tserver with Master: 871aee67d4564a37b4290d09c6257fc2 (127.9.96.65:44843)
WARNING: no policy specified for kudu/127.9.96.66@KRBTEST.COM; defaulting to no policy
I20260806 05:00:28.336572 28147 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.96.65@KRBTEST.COM'} at 127.9.96.65:60581
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.96.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.96.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.66.keytab.
Entry for principal kudu/127.9.96.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.66.keytab.
I20260806 05:00:28.373023 9601 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
/tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785992227050689-9601-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785992227050689-9601-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785992227050689-9601-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785992227050689-9601-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.9.96.84:43097/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.96.66:0
--local_ip_for_outbound_sockets=127.9.96.66
--webserver_interface=127.9.96.66
--webserver_port=0
--tserver_master_addrs=127.9.96.126:45703
--keytab_file=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.66.keytab
--principal=kudu/127.9.96.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.96.84:35379
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785992227050689-9601-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785992227050689-9601-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785992227050689-9601-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 05:00:28 dist-test-slave-x4qh krb5kdc[27396](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992427, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.96.126@KRBTEST.COM for HTTP/127.9.96.84@KRBTEST.COM
Aug 06 05:00:28 dist-test-slave-x4qh krb5kdc[27396](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992427, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.96.126@KRBTEST.COM for HTTP/127.9.96.84@KRBTEST.COM
W20260806 05:00:28.846809 28470 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 05:00:28.847254 28470 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 05:00:28.847488 28470 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 05:00:28.858472 28470 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 05:00:28.858615 28470 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 05:00:28.858873 28470 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.96.66
I20260806 05:00:28.874399 28470 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.96.84:35379
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785992227050689-9601-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785992227050689-9601-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.9.96.84:43097/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.66.keytab
--principal=kudu/127.9.96.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.96.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785992227050689-9601-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.96.66
--webserver_port=0
--tserver_master_addrs=127.9.96.126:45703
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.96.66
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785992227050689-9601-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 03:56:12 UTC on ccycloud.quasar-guijbm.root.comops.site
build id 4569618
ASAN enabled
I20260806 05:00:28.876381 28470 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 05:00:28.878239 28470 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 05:00:28.911937 28485 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 05:00:28.915952 28482 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 05:00:28.921607 28494 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 05:00:28.922151 28470 server_base.cc:1034] running on GCE node
I20260806 05:00:28.922973 28470 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 05:00:28.934458 28470 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 05:00:28.935850 28470 hybrid_clock.cc:648] HybridClock initialized: now 1785992428935729 us; error 116 us; skew 500 ppm
I20260806 05:00:28.936372 28470 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 05:00:28 dist-test-slave-x4qh krb5kdc[27396](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992428, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.96.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 05:00:28.941179 28470 init.cc:377] Logged in from keytab as kudu/127.9.96.66@KRBTEST.COM (short username kudu)
I20260806 05:00:28.961963 28470 webserver.cc:460] Webserver started at http://127.9.96.66:36537/ using document root <none> and password file <none>
I20260806 05:00:28.963223 28470 fs_manager.cc:359] Metadata directory not provided
I20260806 05:00:28.963505 28470 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 05:00:28.964001 28470 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 05:00:28 dist-test-slave-x4qh krb5kdc[27396](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992428, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.96.66@KRBTEST.COM for HTTP/127.9.96.84@KRBTEST.COM
I20260806 05:00:28.991003 28470 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785992227050689-9601-0/minicluster-data/ts-1/data/instance:
uuid: "6d361a41f2774ada80174965738071f4"
format_stamp: "Formatted at 2026-08-06 05:00:28 on dist-test-slave-x4qh"
server_key: "e48a7e8b0e20f2e1673c5b19ca170c6b"
server_key_iv: "6eb607e044ee7aafacd8c6cffbb7b1a7"
server_key_version: "kuduclusterkey@0"
I20260806 05:00:28.992298 28470 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785992227050689-9601-0/minicluster-data/ts-1/wal/instance:
uuid: "6d361a41f2774ada80174965738071f4"
format_stamp: "Formatted at 2026-08-06 05:00:28 on dist-test-slave-x4qh"
server_key: "e48a7e8b0e20f2e1673c5b19ca170c6b"
server_key_iv: "6eb607e044ee7aafacd8c6cffbb7b1a7"
server_key_version: "kuduclusterkey@0"
I20260806 05:00:28.999258 28470 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.001s
I20260806 05:00:29.030443 28503 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 05:00:29.044412 28470 fs_manager.cc:725] Time spent opening block manager: real 0.029s user 0.000s sys 0.016s
I20260806 05:00:29.044653 28470 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785992227050689-9601-0/minicluster-data/ts-1/data,/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785992227050689-9601-0/minicluster-data/ts-1/wal
uuid: "6d361a41f2774ada80174965738071f4"
format_stamp: "Formatted at 2026-08-06 05:00:28 on dist-test-slave-x4qh"
server_key: "e48a7e8b0e20f2e1673c5b19ca170c6b"
server_key_iv: "6eb607e044ee7aafacd8c6cffbb7b1a7"
server_key_version: "kuduclusterkey@0"
I20260806 05:00:29.044919 28470 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785992227050689-9601-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785992227050689-9601-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785992227050689-9601-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 05:00:29.061549 28470 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 05:00:29.066483 28470 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 05:00:29.066821 28470 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 05:00:29.068060 28470 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 05:00:29.069476 28470 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 05:00:29.069585 28470 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 05:00:29.069744 28470 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 05:00:29.069823 28470 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 05:00:29.111768 28470 rpc_server.cc:307] RPC server started. Bound to: 127.9.96.66:46109
I20260806 05:00:29.111786 28700 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.96.66:46109 every 8 connection(s)
I20260806 05:00:29.113188 28470 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785992227050689-9601-0/minicluster-data/ts-1/data/info.pb
I20260806 05:00:29.116366 9601 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu as pid 28470
I20260806 05:00:29.116540 9601 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785992227050689-9601-0/minicluster-data/ts-1/wal/instance
I20260806 05:00:29.117121 9601 external_mini_cluster.cc:1428] Setting key cea054a1240ad8cb4d167133e03d2641
Aug 06 05:00:29 dist-test-slave-x4qh krb5kdc[27396](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992428, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.96.66@KRBTEST.COM for kudu/127.9.96.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.9.96.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.96.67@KRBTEST.COM" created.
I20260806 05:00:29.153837 28705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:00:29.116190 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.96.66:39829 (local address 127.9.96.126:45703)
0806 05:00:29.130906 (+ 14716us) server_negotiation.cc:206] Beginning negotiation
0806 05:00:29.130911 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 05:00:29.134270 (+ 3359us) server_negotiation.cc:407] Connection header received
0806 05:00:29.134353 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 05:00:29.134359 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 05:00:29.134445 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 05:00:29.134570 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0806 05:00:29.136831 (+ 2261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:00:29.137998 (+ 1167us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:00:29.140726 (+ 2728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:00:29.141329 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:00:29.141492 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:00:29.146393 (+ 4901us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 05:00:29.146400 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 05:00:29.146405 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 05:00:29.146450 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0806 05:00:29.150353 (+ 3903us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 05:00:29.151097 (+ 744us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 05:00:29.151103 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 05:00:29.151106 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 05:00:29.151271 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 05:00:29.151876 (+ 605us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 05:00:29.151881 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 05:00:29.151885 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 05:00:29.152293 (+ 408us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 05:00:29.152449 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0806 05:00:29.153224 (+ 775us) server_negotiation.cc:299] Negotiation successful
0806 05:00:29.153454 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14597,"thread_start_us":14474,"threads_started":1}
I20260806 05:00:29.154870 28706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:00:29.116806 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:45703 (local address 127.9.96.66:39829)
0806 05:00:29.131394 (+ 14588us) negotiation.cc:107] Waiting for socket to connect
0806 05:00:29.131465 (+ 71us) client_negotiation.cc:174] Beginning negotiation
0806 05:00:29.133896 (+ 2431us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 05:00:29.134937 (+ 1041us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 05:00:29.134956 (+ 19us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 05:00:29.135677 (+ 721us) client_negotiation.cc:189] Negotiated authn=SASL
0806 05:00:29.136582 (+ 905us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 05:00:29.136617 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 05:00:29.138181 (+ 1564us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 05:00:29.138187 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 05:00:29.140556 (+ 2369us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 05:00:29.140574 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 05:00:29.141632 (+ 1058us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 05:00:29.141645 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 05:00:29.141997 (+ 352us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:00:29.142839 (+ 842us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 05:00:29.142875 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0806 05:00:29.146095 (+ 3220us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 05:00:29.150494 (+ 4399us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 05:00:29.150501 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 05:00:29.150514 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0806 05:00:29.150960 (+ 446us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 05:00:29.151458 (+ 498us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 05:00:29.151465 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 05:00:29.151472 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0806 05:00:29.151749 (+ 277us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 05:00:29.152449 (+ 700us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 05:00:29.152460 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 05:00:29.152844 (+ 384us) client_negotiation.cc:769] Sending connection context
0806 05:00:29.153121 (+ 277us) client_negotiation.cc:240] Negotiation successful
0806 05:00:29.153656 (+ 535us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14235,"thread_start_us":13970,"threads_started":1}
I20260806 05:00:29.156394 28703 heartbeater.cc:347] Connected to a master server at 127.9.96.126:45703
I20260806 05:00:29.156714 28703 heartbeater.cc:464] Registering TS with master...
I20260806 05:00:29.157438 28703 heartbeater.cc:511] Master 127.9.96.126:45703 requested a full tablet report, sending...
I20260806 05:00:29.158916 28147 ts_manager.cc:194] Registered new tserver with Master: 6d361a41f2774ada80174965738071f4 (127.9.96.66:46109)
I20260806 05:00:29.159770 28147 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.96.66@KRBTEST.COM'} at 127.9.96.66:39829
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.96.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.67.keytab.
Entry for principal kudu/127.9.96.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.67.keytab.
I20260806 05:00:29.188974 9601 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
/tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785992227050689-9601-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785992227050689-9601-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785992227050689-9601-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785992227050689-9601-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.9.96.84:43097/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.96.67:0
--local_ip_for_outbound_sockets=127.9.96.67
--webserver_interface=127.9.96.67
--webserver_port=0
--tserver_master_addrs=127.9.96.126:45703
--keytab_file=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.67.keytab
--principal=kudu/127.9.96.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.96.84:35379
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785992227050689-9601-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785992227050689-9601-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785992227050689-9601-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 05:00:29.340469 28463 heartbeater.cc:503] Master 127.9.96.126:45703 was elected leader, sending a full tablet report...
W20260806 05:00:29.553715 28713 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 05:00:29.554051 28713 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 05:00:29.554221 28713 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 05:00:29.561430 28713 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 05:00:29.561584 28713 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 05:00:29.561781 28713 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.96.67
I20260806 05:00:29.571410 28713 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.96.84:35379
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785992227050689-9601-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785992227050689-9601-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.9.96.84:43097/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.67.keytab
--principal=kudu/127.9.96.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.96.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785992227050689-9601-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.96.67
--webserver_port=0
--tserver_master_addrs=127.9.96.126:45703
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.96.67
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785992227050689-9601-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 03:56:12 UTC on ccycloud.quasar-guijbm.root.comops.site
build id 4569618
ASAN enabled
I20260806 05:00:29.572664 28713 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 05:00:29.573939 28713 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 05:00:29.587651 28724 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 05:00:29.587735 28721 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 05:00:29.587711 28722 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 05:00:29.588611 28713 server_base.cc:1034] running on GCE node
I20260806 05:00:29.589218 28713 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 05:00:29.590189 28713 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 05:00:29.591475 28713 hybrid_clock.cc:648] HybridClock initialized: now 1785992429591379 us; error 84 us; skew 500 ppm
I20260806 05:00:29.591792 28713 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 05:00:29 dist-test-slave-x4qh krb5kdc[27396](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992429, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.96.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 05:00:29.594913 28713 init.cc:377] Logged in from keytab as kudu/127.9.96.67@KRBTEST.COM (short username kudu)
I20260806 05:00:29.596343 28713 webserver.cc:460] Webserver started at http://127.9.96.67:36413/ using document root <none> and password file <none>
I20260806 05:00:29.597033 28713 fs_manager.cc:359] Metadata directory not provided
I20260806 05:00:29.597180 28713 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 05:00:29.597517 28713 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 05:00:29 dist-test-slave-x4qh krb5kdc[27396](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992429, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.96.67@KRBTEST.COM for HTTP/127.9.96.84@KRBTEST.COM
I20260806 05:00:29.613194 28713 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785992227050689-9601-0/minicluster-data/ts-2/data/instance:
uuid: "0d27ed03047840a6bf076799cf5d5cd7"
format_stamp: "Formatted at 2026-08-06 05:00:29 on dist-test-slave-x4qh"
server_key: "fb2c155ce6b2e54dbb6a98ce8ed35fe3"
server_key_iv: "290ed9e2629c239ee383a90057f4c44b"
server_key_version: "kuduclusterkey@0"
I20260806 05:00:29.614084 28713 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785992227050689-9601-0/minicluster-data/ts-2/wal/instance:
uuid: "0d27ed03047840a6bf076799cf5d5cd7"
format_stamp: "Formatted at 2026-08-06 05:00:29 on dist-test-slave-x4qh"
server_key: "fb2c155ce6b2e54dbb6a98ce8ed35fe3"
server_key_iv: "290ed9e2629c239ee383a90057f4c44b"
server_key_version: "kuduclusterkey@0"
I20260806 05:00:29.619454 28713 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260806 05:00:29.635874 28736 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 05:00:29.637573 28713 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.003s
I20260806 05:00:29.637753 28713 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785992227050689-9601-0/minicluster-data/ts-2/data,/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785992227050689-9601-0/minicluster-data/ts-2/wal
uuid: "0d27ed03047840a6bf076799cf5d5cd7"
format_stamp: "Formatted at 2026-08-06 05:00:29 on dist-test-slave-x4qh"
server_key: "fb2c155ce6b2e54dbb6a98ce8ed35fe3"
server_key_iv: "290ed9e2629c239ee383a90057f4c44b"
server_key_version: "kuduclusterkey@0"
I20260806 05:00:29.638003 28713 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785992227050689-9601-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785992227050689-9601-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785992227050689-9601-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 05:00:29.658039 28713 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 05:00:29.661561 28713 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 05:00:29.661942 28713 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 05:00:29.662916 28713 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 05:00:29.664297 28713 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 05:00:29.664373 28713 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 05:00:29.664480 28713 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 05:00:29.664525 28713 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 05:00:29.698549 28713 rpc_server.cc:307] RPC server started. Bound to: 127.9.96.67:39039
I20260806 05:00:29.698611 28927 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.96.67:39039 every 8 connection(s)
I20260806 05:00:29.699970 28713 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785992227050689-9601-0/minicluster-data/ts-2/data/info.pb
I20260806 05:00:29.709980 9601 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu as pid 28713
I20260806 05:00:29.710157 9601 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785992227050689-9601-0/minicluster-data/ts-2/wal/instance
I20260806 05:00:29.710636 9601 external_mini_cluster.cc:1428] Setting key d1063f76cc98cf679140b2e4a4f975c9
Aug 06 05:00:29 dist-test-slave-x4qh krb5kdc[27396](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992429, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.96.67@KRBTEST.COM for kudu/127.9.96.126@KRBTEST.COM
I20260806 05:00:29.719277 28933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:00:29.702537 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.96.67:59451 (local address 127.9.96.126:45703)
0806 05:00:29.703000 (+ 463us) server_negotiation.cc:206] Beginning negotiation
0806 05:00:29.703004 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 05:00:29.703992 (+ 988us) server_negotiation.cc:407] Connection header received
0806 05:00:29.705755 (+ 1763us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 05:00:29.705761 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 05:00:29.705864 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 05:00:29.705987 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0806 05:00:29.707363 (+ 1376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:00:29.708124 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:00:29.709686 (+ 1562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:00:29.710217 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:00:29.710368 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:00:29.714140 (+ 3772us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 05:00:29.714145 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 05:00:29.714149 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 05:00:29.714181 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0806 05:00:29.716634 (+ 2453us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 05:00:29.717412 (+ 778us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 05:00:29.717416 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 05:00:29.717418 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 05:00:29.717542 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 05:00:29.717910 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 05:00:29.717913 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 05:00:29.717915 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 05:00:29.718174 (+ 259us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 05:00:29.718284 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0806 05:00:29.718857 (+ 573us) server_negotiation.cc:299] Negotiation successful
0806 05:00:29.719029 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":359,"thread_start_us":215,"threads_started":1}
I20260806 05:00:29.720288 28934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:00:29.703083 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:45703 (local address 127.9.96.67:59451)
0806 05:00:29.703843 (+ 760us) negotiation.cc:107] Waiting for socket to connect
0806 05:00:29.703880 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0806 05:00:29.705491 (+ 1611us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 05:00:29.706222 (+ 731us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 05:00:29.706230 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 05:00:29.706643 (+ 413us) client_negotiation.cc:189] Negotiated authn=SASL
0806 05:00:29.707190 (+ 547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 05:00:29.707206 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 05:00:29.708289 (+ 1083us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 05:00:29.708294 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 05:00:29.709514 (+ 1220us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 05:00:29.709526 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 05:00:29.710454 (+ 928us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 05:00:29.710460 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 05:00:29.710735 (+ 275us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:00:29.711491 (+ 756us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 05:00:29.711513 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0806 05:00:29.713964 (+ 2451us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 05:00:29.716793 (+ 2829us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 05:00:29.716811 (+ 18us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 05:00:29.716825 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0806 05:00:29.717311 (+ 486us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 05:00:29.717635 (+ 324us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 05:00:29.717638 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 05:00:29.717640 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 05:00:29.717817 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 05:00:29.718276 (+ 459us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 05:00:29.718281 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 05:00:29.718568 (+ 287us) client_negotiation.cc:769] Sending connection context
0806 05:00:29.718783 (+ 215us) client_negotiation.cc:240] Negotiation successful
0806 05:00:29.719150 (+ 367us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":527,"thread_start_us":342,"threads_started":1}
I20260806 05:00:29.721649 28930 heartbeater.cc:347] Connected to a master server at 127.9.96.126:45703
I20260806 05:00:29.721944 28930 heartbeater.cc:464] Registering TS with master...
I20260806 05:00:29.722643 28930 heartbeater.cc:511] Master 127.9.96.126:45703 requested a full tablet report, sending...
I20260806 05:00:29.724145 28147 ts_manager.cc:194] Registered new tserver with Master: 0d27ed03047840a6bf076799cf5d5cd7 (127.9.96.67:39039)
I20260806 05:00:29.725310 28147 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.96.67@KRBTEST.COM'} at 127.9.96.67:59451
I20260806 05:00:29.726235 9601 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 05:00:29.728528 9601 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu pid 28262
Aug 06 05:00:29 dist-test-slave-x4qh krb5kdc[27396](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992426, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.96.65@KRBTEST.COM
I20260806 05:00:29.745452 28937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:00:29.729491 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60202 (local address 127.9.96.65:44843)
0806 05:00:29.730169 (+ 678us) server_negotiation.cc:206] Beginning negotiation
0806 05:00:29.730175 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 05:00:29.730197 (+ 22us) server_negotiation.cc:407] Connection header received
0806 05:00:29.730251 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 05:00:29.730256 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 05:00:29.730425 (+ 169us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 05:00:29.730592 (+ 167us) server_negotiation.cc:226] Negotiated authn=SASL
0806 05:00:29.732196 (+ 1604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:00:29.733178 (+ 982us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:00:29.734768 (+ 1590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:00:29.735325 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:00:29.735514 (+ 189us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:00:29.739724 (+ 4210us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 05:00:29.739736 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 05:00:29.739748 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 05:00:29.739780 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0806 05:00:29.742383 (+ 2603us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 05:00:29.743209 (+ 826us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 05:00:29.743224 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 05:00:29.743226 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 05:00:29.743434 (+ 208us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 05:00:29.743851 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 05:00:29.743856 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 05:00:29.743860 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 05:00:29.744186 (+ 326us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 05:00:29.744517 (+ 331us) server_negotiation.cc:1035] Waiting for connection context
0806 05:00:29.744829 (+ 312us) server_negotiation.cc:299] Negotiation successful
0806 05:00:29.745102 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":494,"thread_start_us":330,"threads_started":1}
I20260806 05:00:29.746734 28457 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 05:00:29.829823 9601 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu with pid 28262
I20260806 05:00:29.854174 9601 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu pid 28470
Aug 06 05:00:29 dist-test-slave-x4qh krb5kdc[27396](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992426, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.96.66@KRBTEST.COM
I20260806 05:00:29.875231 28941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:00:29.855331 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47512 (local address 127.9.96.66:46109)
0806 05:00:29.856047 (+ 716us) server_negotiation.cc:206] Beginning negotiation
0806 05:00:29.856054 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 05:00:29.856080 (+ 26us) server_negotiation.cc:407] Connection header received
0806 05:00:29.856141 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 05:00:29.856147 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 05:00:29.856362 (+ 215us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 05:00:29.856489 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0806 05:00:29.857706 (+ 1217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:00:29.858708 (+ 1002us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:00:29.860028 (+ 1320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:00:29.860562 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:00:29.860661 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:00:29.869632 (+ 8971us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 05:00:29.869644 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 05:00:29.869659 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 05:00:29.869702 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0806 05:00:29.872715 (+ 3013us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 05:00:29.873330 (+ 615us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 05:00:29.873337 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 05:00:29.873339 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 05:00:29.873480 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 05:00:29.873804 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 05:00:29.873807 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 05:00:29.873809 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 05:00:29.874146 (+ 337us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 05:00:29.874350 (+ 204us) server_negotiation.cc:1035] Waiting for connection context
0806 05:00:29.874605 (+ 255us) server_negotiation.cc:299] Negotiation successful
0806 05:00:29.874805 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":573,"thread_start_us":350,"threads_started":1}
I20260806 05:00:29.876559 28692 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 05:00:29.958068 9601 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu with pid 28470
I20260806 05:00:29.979339 9601 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu pid 28713
Aug 06 05:00:29 dist-test-slave-x4qh krb5kdc[27396](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992426, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.96.67@KRBTEST.COM
I20260806 05:00:29.995349 28947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:00:29.980482 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54006 (local address 127.9.96.67:39039)
0806 05:00:29.981102 (+ 620us) server_negotiation.cc:206] Beginning negotiation
0806 05:00:29.981109 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 05:00:29.981137 (+ 28us) server_negotiation.cc:407] Connection header received
0806 05:00:29.981196 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 05:00:29.981202 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 05:00:29.981432 (+ 230us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 05:00:29.981578 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0806 05:00:29.982926 (+ 1348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:00:29.984008 (+ 1082us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:00:29.985247 (+ 1239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:00:29.985797 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:00:29.985893 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:00:29.990118 (+ 4225us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 05:00:29.990130 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 05:00:29.990145 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 05:00:29.990179 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0806 05:00:29.992878 (+ 2699us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 05:00:29.993503 (+ 625us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 05:00:29.993510 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 05:00:29.993512 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 05:00:29.993631 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 05:00:29.993965 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 05:00:29.993968 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 05:00:29.993970 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 05:00:29.994285 (+ 315us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 05:00:29.994567 (+ 282us) server_negotiation.cc:1035] Waiting for connection context
0806 05:00:29.994859 (+ 292us) server_negotiation.cc:299] Negotiation successful
0806 05:00:29.995051 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":475,"thread_start_us":319,"threads_started":1}
I20260806 05:00:29.996757 28910 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 05:00:30.082013 9601 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu with pid 28713
I20260806 05:00:30.105079 9601 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu pid 28094
I20260806 05:00:30.106333 28192 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 05:00:30.211491 9601 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu with pid 28094
I20260806 05:00:30.245326 9601 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
/tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785992227050689-9601-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785992227050689-9601-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785992227050689-9601-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785992227050689-9601-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.96.126:45703
--webserver_interface=127.9.96.126
--webserver_port=34245
--keytab_file=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.126.keytab
--principal=kudu/127.9.96.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.96.84:35379
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.96.126:45703
--ranger_config_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785992227050689-9601-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.9.96.84:43097/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.9.96.84:43097/kms with env {KRB5CCNAME=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785992227050689-9601-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785992227050689-9601-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785992227050689-9601-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 05:00:30.576117 28950 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 05:00:30.576388 28950 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 05:00:30.576478 28950 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 05:00:30.583513 28950 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 05:00:30.583616 28950 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 05:00:30.583663 28950 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 05:00:30.583720 28950 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 05:00:30.583771 28950 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 05:00:30.593545 28950 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.96.84:35379
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785992227050689-9601-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785992227050689-9601-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.9.96.84:43097/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.9.96.126:45703
--ranger_config_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785992227050689-9601-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.126.keytab
--principal=kudu/127.9.96.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.96.126:45703
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785992227050689-9601-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.96.126
--webserver_port=34245
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785992227050689-9601-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 03:56:12 UTC on ccycloud.quasar-guijbm.root.comops.site
build id 4569618
ASAN enabled
I20260806 05:00:30.594666 28950 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 05:00:30.596087 28950 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 05:00:30.608052 28961 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 05:00:30.608085 28963 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 05:00:30.608129 28960 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 05:00:30.608882 28950 server_base.cc:1034] running on GCE node
I20260806 05:00:30.609632 28950 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 05:00:30.611017 28950 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 05:00:30.612222 28950 hybrid_clock.cc:648] HybridClock initialized: now 1785992430612185 us; error 29 us; skew 500 ppm
I20260806 05:00:30.612566 28950 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 05:00:30 dist-test-slave-x4qh krb5kdc[27396](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992430, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.96.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 05:00:30.615723 28950 init.cc:377] Logged in from keytab as kudu/127.9.96.126@KRBTEST.COM (short username kudu)
I20260806 05:00:30.617195 28950 webserver.cc:460] Webserver started at http://127.9.96.126:34245/ using document root <none> and password file <none>
I20260806 05:00:30.617843 28950 fs_manager.cc:359] Metadata directory not provided
I20260806 05:00:30.618008 28950 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 06 05:00:30 dist-test-slave-x4qh krb5kdc[27396](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992430, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.96.126@KRBTEST.COM for HTTP/127.9.96.84@KRBTEST.COM
I20260806 05:00:30.671334 28950 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260806 05:00:30.674254 28977 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 05:00:30.675702 28950 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 05:00:30.675906 28950 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785992227050689-9601-0/minicluster-data/master-0/data,/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785992227050689-9601-0/minicluster-data/master-0/wal
uuid: "57daa83eda8242b5af95a57caa6b61a8"
format_stamp: "Formatted at 2026-08-06 05:00:26 on dist-test-slave-x4qh"
server_key: "f6c9746c3871c953a32a97a1f4c7c890"
server_key_iv: "569c07ed3d500e6d3e9b613d842af8d8"
server_key_version: "kuduclusterkey@0"
I20260806 05:00:30.676868 28950 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785992227050689-9601-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785992227050689-9601-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785992227050689-9601-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 05:00:30.692567 28950 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 05:00:30.695977 28950 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 05:00:30.696316 28950 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 05:00:30.715498 28950 rpc_server.cc:307] RPC server started. Bound to: 127.9.96.126:45703
I20260806 05:00:30.715543 29072 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.96.126:45703 every 8 connection(s)
I20260806 05:00:30.717080 28950 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785992227050689-9601-0/minicluster-data/master-0/data/info.pb
I20260806 05:00:30.720602 9601 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu as pid 28950
I20260806 05:00:30.721486 9601 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
/tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785992227050689-9601-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785992227050689-9601-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785992227050689-9601-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785992227050689-9601-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.9.96.84:43097/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.96.65:44843
--local_ip_for_outbound_sockets=127.9.96.65
--tserver_master_addrs=127.9.96.126:45703
--webserver_port=44839
--webserver_interface=127.9.96.65
--keytab_file=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.65.keytab
--principal=kudu/127.9.96.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.96.84:35379
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.9.96.84:43097/kms with env {KRB5CCNAME=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785992227050689-9601-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785992227050689-9601-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785992227050689-9601-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 05:00:30.724157 29073 sys_catalog.cc:263] Verifying existing consensus state
I20260806 05:00:30.727502 29073 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 57daa83eda8242b5af95a57caa6b61a8: Bootstrap starting.
I20260806 05:00:30.752663 29073 log.cc:824] T 00000000000000000000000000000000 P 57daa83eda8242b5af95a57caa6b61a8: Log is configured to *not* fsync() on all Append() calls
I20260806 05:00:30.761629 29073 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 57daa83eda8242b5af95a57caa6b61a8: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 05:00:30.762161 29073 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 57daa83eda8242b5af95a57caa6b61a8: Bootstrap complete.
I20260806 05:00:30.771143 29073 raft_consensus.cc:348] T 00000000000000000000000000000000 P 57daa83eda8242b5af95a57caa6b61a8 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57daa83eda8242b5af95a57caa6b61a8" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 45703 } }
I20260806 05:00:30.772085 29073 raft_consensus.cc:729] T 00000000000000000000000000000000 P 57daa83eda8242b5af95a57caa6b61a8 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57daa83eda8242b5af95a57caa6b61a8, State: Initialized, Role: FOLLOWER
I20260806 05:00:30.772730 29073 consensus_queue.cc:260] T 00000000000000000000000000000000 P 57daa83eda8242b5af95a57caa6b61a8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57daa83eda8242b5af95a57caa6b61a8" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 45703 } }
I20260806 05:00:30.772977 29073 raft_consensus.cc:388] T 00000000000000000000000000000000 P 57daa83eda8242b5af95a57caa6b61a8 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 05:00:30.773100 29073 raft_consensus.cc:482] T 00000000000000000000000000000000 P 57daa83eda8242b5af95a57caa6b61a8 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 05:00:30.773281 29073 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 57daa83eda8242b5af95a57caa6b61a8 [term 1 FOLLOWER]: Advancing to term 2
I20260806 05:00:30.777170 29073 raft_consensus.cc:504] T 00000000000000000000000000000000 P 57daa83eda8242b5af95a57caa6b61a8 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57daa83eda8242b5af95a57caa6b61a8" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 45703 } }
I20260806 05:00:30.777686 29073 leader_election.cc:302] T 00000000000000000000000000000000 P 57daa83eda8242b5af95a57caa6b61a8 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 57daa83eda8242b5af95a57caa6b61a8; no voters:
I20260806 05:00:30.778347 29073 leader_election.cc:288] T 00000000000000000000000000000000 P 57daa83eda8242b5af95a57caa6b61a8 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260806 05:00:30.778481 29077 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 57daa83eda8242b5af95a57caa6b61a8 [term 2 FOLLOWER]: Leader election won for term 2
I20260806 05:00:30.779747 29077 raft_consensus.cc:686] T 00000000000000000000000000000000 P 57daa83eda8242b5af95a57caa6b61a8 [term 2 LEADER]: Becoming Leader. State: Replica: 57daa83eda8242b5af95a57caa6b61a8, State: Running, Role: LEADER
I20260806 05:00:30.780346 29077 consensus_queue.cc:237] T 00000000000000000000000000000000 P 57daa83eda8242b5af95a57caa6b61a8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57daa83eda8242b5af95a57caa6b61a8" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 45703 } }
I20260806 05:00:30.780684 29073 sys_catalog.cc:564] T 00000000000000000000000000000000 P 57daa83eda8242b5af95a57caa6b61a8 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 05:00:30.783010 29080 sys_catalog.cc:455] T 00000000000000000000000000000000 P 57daa83eda8242b5af95a57caa6b61a8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 57daa83eda8242b5af95a57caa6b61a8. Latest consensus state: current_term: 2 leader_uuid: "57daa83eda8242b5af95a57caa6b61a8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57daa83eda8242b5af95a57caa6b61a8" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 45703 } } }
I20260806 05:00:30.783504 29080 sys_catalog.cc:458] T 00000000000000000000000000000000 P 57daa83eda8242b5af95a57caa6b61a8 [sys.catalog]: This master's current role is: LEADER
I20260806 05:00:30.784139 29079 sys_catalog.cc:455] T 00000000000000000000000000000000 P 57daa83eda8242b5af95a57caa6b61a8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "57daa83eda8242b5af95a57caa6b61a8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57daa83eda8242b5af95a57caa6b61a8" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 45703 } } }
I20260806 05:00:30.784488 29079 sys_catalog.cc:458] T 00000000000000000000000000000000 P 57daa83eda8242b5af95a57caa6b61a8 [sys.catalog]: This master's current role is: LEADER
I20260806 05:00:30.787683 29092 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 05:00:30.794498 29092 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 05:00:30.797434 29092 catalog_manager.cc:1259] Loaded cluster ID: 5a864725429f4e8082b1687f5beb9217
I20260806 05:00:30.797715 29092 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 05:00:30.803755 29092 catalog_manager.cc:1504] Loading token signing keys...
I20260806 05:00:30.804360 29073 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785992227050689-9601-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260806 05:00:30.805332 29092 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 57daa83eda8242b5af95a57caa6b61a8: Loaded TSK: 0
I20260806 05:00:30.812978 29092 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260806 05:00:31.185211 29075 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 05:00:31.185621 29075 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 05:00:31.185869 29075 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 05:00:31.194983 29075 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 05:00:31.195185 29075 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 05:00:31.195446 29075 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.96.65
I20260806 05:00:31.207684 29075 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.96.84:35379
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785992227050689-9601-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785992227050689-9601-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.9.96.84:43097/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.65.keytab
--principal=kudu/127.9.96.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.96.65:44843
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785992227050689-9601-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.96.65
--webserver_port=44839
--tserver_master_addrs=127.9.96.126:45703
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.96.65
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785992227050689-9601-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 03:56:12 UTC on ccycloud.quasar-guijbm.root.comops.site
build id 4569618
ASAN enabled
I20260806 05:00:31.209434 29075 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 05:00:31.211266 29075 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 05:00:31.252751 29136 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 05:00:31.254016 29134 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 05:00:31.254133 29133 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 05:00:31.255985 29075 server_base.cc:1034] running on GCE node
I20260806 05:00:31.256943 29075 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 05:00:31.262007 29075 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 05:00:31.263346 29075 hybrid_clock.cc:648] HybridClock initialized: now 1785992431263263 us; error 64 us; skew 500 ppm
I20260806 05:00:31.263909 29075 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 05:00:31 dist-test-slave-x4qh krb5kdc[27396](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992431, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.96.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 05:00:31.267277 29075 init.cc:377] Logged in from keytab as kudu/127.9.96.65@KRBTEST.COM (short username kudu)
I20260806 05:00:31.269296 29075 webserver.cc:460] Webserver started at http://127.9.96.65:44839/ using document root <none> and password file <none>
I20260806 05:00:31.270013 29075 fs_manager.cc:359] Metadata directory not provided
I20260806 05:00:31.270305 29075 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 06 05:00:31 dist-test-slave-x4qh krb5kdc[27396](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992431, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.96.65@KRBTEST.COM for HTTP/127.9.96.84@KRBTEST.COM
I20260806 05:00:31.295892 29075 fs_manager.cc:709] Time spent opening directory manager: real 0.010s user 0.005s sys 0.004s
I20260806 05:00:31.299697 29152 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 05:00:31.301669 29075 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260806 05:00:31.301872 29075 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785992227050689-9601-0/minicluster-data/ts-0/data,/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785992227050689-9601-0/minicluster-data/ts-0/wal
uuid: "871aee67d4564a37b4290d09c6257fc2"
format_stamp: "Formatted at 2026-08-06 05:00:28 on dist-test-slave-x4qh"
server_key: "43f2b9b9fef214cc19369c5b848fc940"
server_key_iv: "de9ff71b8a3b4360feb82be6355b846c"
server_key_version: "kuduclusterkey@0"
I20260806 05:00:31.303300 29075 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785992227050689-9601-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785992227050689-9601-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785992227050689-9601-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 05:00:31.333544 29075 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 05:00:31.338992 29075 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 05:00:31.339419 29075 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 05:00:31.343825 29075 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 05:00:31.345551 29075 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 05:00:31.345682 29075 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 05:00:31.345829 29075 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 05:00:31.345938 29075 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 05:00:31.448444 29075 rpc_server.cc:307] RPC server started. Bound to: 127.9.96.65:44843
I20260806 05:00:31.448482 29341 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.96.65:44843 every 8 connection(s)
I20260806 05:00:31.450479 29075 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785992227050689-9601-0/minicluster-data/ts-0/data/info.pb
I20260806 05:00:31.458598 9601 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu as pid 29075
I20260806 05:00:31.459537 9601 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
/tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785992227050689-9601-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785992227050689-9601-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785992227050689-9601-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785992227050689-9601-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.9.96.84:43097/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.96.66:46109
--local_ip_for_outbound_sockets=127.9.96.66
--tserver_master_addrs=127.9.96.126:45703
--webserver_port=36537
--webserver_interface=127.9.96.66
--keytab_file=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.66.keytab
--principal=kudu/127.9.96.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.96.84:35379
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.9.96.84:43097/kms with env {KRB5CCNAME=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785992227050689-9601-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785992227050689-9601-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785992227050689-9601-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 05:00:31 dist-test-slave-x4qh krb5kdc[27396](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992431, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.96.65@KRBTEST.COM for kudu/127.9.96.126@KRBTEST.COM
I20260806 05:00:31.482312 29347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:00:31.454040 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:45703 (local address 127.9.96.65:57613)
0806 05:00:31.455892 (+ 1852us) negotiation.cc:107] Waiting for socket to connect
0806 05:00:31.455931 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0806 05:00:31.457573 (+ 1642us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 05:00:31.460055 (+ 2482us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 05:00:31.460067 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 05:00:31.460726 (+ 659us) client_negotiation.cc:189] Negotiated authn=SASL
0806 05:00:31.461483 (+ 757us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 05:00:31.461506 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 05:00:31.463954 (+ 2448us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 05:00:31.463961 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 05:00:31.466185 (+ 2224us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 05:00:31.466226 (+ 41us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 05:00:31.467716 (+ 1490us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 05:00:31.467726 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 05:00:31.468040 (+ 314us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:00:31.468837 (+ 797us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 05:00:31.468871 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0806 05:00:31.471875 (+ 3004us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 05:00:31.476724 (+ 4849us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 05:00:31.476735 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 05:00:31.476752 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0806 05:00:31.477448 (+ 696us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 05:00:31.477996 (+ 548us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 05:00:31.478004 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 05:00:31.478010 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0806 05:00:31.478286 (+ 276us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 05:00:31.479346 (+ 1060us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 05:00:31.479357 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 05:00:31.479819 (+ 462us) client_negotiation.cc:769] Sending connection context
0806 05:00:31.480214 (+ 395us) client_negotiation.cc:240] Negotiation successful
0806 05:00:31.480819 (+ 605us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1609,"thread_start_us":1406,"threads_started":1}
I20260806 05:00:31.485248 29346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:00:31.454023 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.96.65:57613 (local address 127.9.96.126:45703)
0806 05:00:31.458380 (+ 4357us) server_negotiation.cc:206] Beginning negotiation
0806 05:00:31.458400 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0806 05:00:31.458452 (+ 52us) server_negotiation.cc:407] Connection header received
0806 05:00:31.458913 (+ 461us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 05:00:31.458930 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 05:00:31.459443 (+ 513us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 05:00:31.459869 (+ 426us) server_negotiation.cc:226] Negotiated authn=SASL
0806 05:00:31.461726 (+ 1857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:00:31.463741 (+ 2015us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:00:31.466386 (+ 2645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:00:31.467471 (+ 1085us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:00:31.467694 (+ 223us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:00:31.472178 (+ 4484us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 05:00:31.472191 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 05:00:31.472223 (+ 32us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 05:00:31.472274 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0806 05:00:31.476561 (+ 4287us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 05:00:31.477642 (+ 1081us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 05:00:31.477653 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 05:00:31.477663 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0806 05:00:31.477861 (+ 198us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 05:00:31.478454 (+ 593us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 05:00:31.478461 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 05:00:31.478467 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 05:00:31.479174 (+ 707us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 05:00:31.483162 (+ 3988us) server_negotiation.cc:1035] Waiting for connection context
0806 05:00:31.483477 (+ 315us) server_negotiation.cc:299] Negotiation successful
0806 05:00:31.483966 (+ 489us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4054,"thread_start_us":3844,"threads_started":1}
I20260806 05:00:31.488965 29344 heartbeater.cc:347] Connected to a master server at 127.9.96.126:45703
I20260806 05:00:31.489319 29344 heartbeater.cc:464] Registering TS with master...
I20260806 05:00:31.490274 29344 heartbeater.cc:511] Master 127.9.96.126:45703 requested a full tablet report, sending...
W20260806 05:00:31.493863 29344 heartbeater.cc:644] Failed to heartbeat to 127.9.96.126:45703 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260806 05:00:31.494086 29344 heartbeater.cc:464] Registering TS with master...
I20260806 05:00:31.494359 29344 heartbeater.cc:511] Master 127.9.96.126:45703 requested a full tablet report, sending...
I20260806 05:00:31.495595 29344 heartbeater.cc:464] Registering TS with master...
I20260806 05:00:31.495860 29344 heartbeater.cc:511] Master 127.9.96.126:45703 requested a full tablet report, sending...
W20260806 05:00:31.496918 29344 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260806 05:00:31.888664 29349 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 05:00:31.888999 29349 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 05:00:31.889168 29349 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 05:00:31.896682 29349 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 05:00:31.896888 29349 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 05:00:31.897131 29349 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.96.66
I20260806 05:00:31.908895 29349 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.96.84:35379
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785992227050689-9601-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785992227050689-9601-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.9.96.84:43097/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.66.keytab
--principal=kudu/127.9.96.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.96.66:46109
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785992227050689-9601-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.96.66
--webserver_port=36537
--tserver_master_addrs=127.9.96.126:45703
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.96.66
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785992227050689-9601-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 03:56:12 UTC on ccycloud.quasar-guijbm.root.comops.site
build id 4569618
ASAN enabled
I20260806 05:00:31.910373 29349 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 05:00:31.911759 29349 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 05:00:31.926343 29370 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 05:00:31.926347 29368 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 05:00:31.927209 29373 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 05:00:31.927649 29349 server_base.cc:1034] running on GCE node
I20260806 05:00:31.928299 29349 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 05:00:31.929505 29349 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 05:00:31.930760 29349 hybrid_clock.cc:648] HybridClock initialized: now 1785992431930686 us; error 55 us; skew 500 ppm
I20260806 05:00:31.931176 29349 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 05:00:31 dist-test-slave-x4qh krb5kdc[27396](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992431, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.96.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 05:00:31.935436 29349 init.cc:377] Logged in from keytab as kudu/127.9.96.66@KRBTEST.COM (short username kudu)
I20260806 05:00:31.937156 29349 webserver.cc:460] Webserver started at http://127.9.96.66:36537/ using document root <none> and password file <none>
I20260806 05:00:31.937999 29349 fs_manager.cc:359] Metadata directory not provided
I20260806 05:00:31.938179 29349 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 06 05:00:31 dist-test-slave-x4qh krb5kdc[27396](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992431, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.96.66@KRBTEST.COM for HTTP/127.9.96.84@KRBTEST.COM
I20260806 05:00:31.980515 29349 fs_manager.cc:709] Time spent opening directory manager: real 0.017s user 0.009s sys 0.001s
I20260806 05:00:31.984447 29388 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 05:00:31.986337 29349 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260806 05:00:31.986532 29349 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785992227050689-9601-0/minicluster-data/ts-1/data,/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785992227050689-9601-0/minicluster-data/ts-1/wal
uuid: "6d361a41f2774ada80174965738071f4"
format_stamp: "Formatted at 2026-08-06 05:00:28 on dist-test-slave-x4qh"
server_key: "e48a7e8b0e20f2e1673c5b19ca170c6b"
server_key_iv: "6eb607e044ee7aafacd8c6cffbb7b1a7"
server_key_version: "kuduclusterkey@0"
I20260806 05:00:31.988065 29349 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785992227050689-9601-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785992227050689-9601-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785992227050689-9601-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 05:00:32.016145 29349 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 05:00:32.021559 29349 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 05:00:32.021878 29349 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 05:00:32.038436 29349 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 05:00:32.041625 29349 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 05:00:32.041754 29349 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 05:00:32.041886 29349 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 05:00:32.041975 29349 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 05:00:32.116230 29349 rpc_server.cc:307] RPC server started. Bound to: 127.9.96.66:46109
I20260806 05:00:32.116276 29561 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.96.66:46109 every 8 connection(s)
I20260806 05:00:32.117781 29349 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785992227050689-9601-0/minicluster-data/ts-1/data/info.pb
I20260806 05:00:32.127905 9601 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu as pid 29349
I20260806 05:00:32.128917 9601 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
/tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785992227050689-9601-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785992227050689-9601-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785992227050689-9601-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785992227050689-9601-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.9.96.84:43097/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.96.67:39039
--local_ip_for_outbound_sockets=127.9.96.67
--tserver_master_addrs=127.9.96.126:45703
--webserver_port=36413
--webserver_interface=127.9.96.67
--keytab_file=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.67.keytab
--principal=kudu/127.9.96.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.96.84:35379
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.9.96.84:43097/kms with env {KRB5CCNAME=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785992227050689-9601-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785992227050689-9601-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785992227050689-9601-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 05:00:32 dist-test-slave-x4qh krb5kdc[27396](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1785992432, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.96.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Aug 06 05:00:32 dist-test-slave-x4qh krb5kdc[27396](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992431, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.96.66@KRBTEST.COM for kudu/127.9.96.126@KRBTEST.COM
I20260806 05:00:32.174356 29566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:00:32.123868 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.96.66:37669 (local address 127.9.96.126:45703)
0806 05:00:32.151979 (+ 28111us) server_negotiation.cc:206] Beginning negotiation
0806 05:00:32.151988 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 05:00:32.152011 (+ 23us) server_negotiation.cc:407] Connection header received
0806 05:00:32.152078 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 05:00:32.152086 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 05:00:32.152188 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 05:00:32.152297 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0806 05:00:32.154668 (+ 2371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:00:32.155988 (+ 1320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:00:32.158778 (+ 2790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:00:32.159633 (+ 855us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:00:32.159753 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:00:32.165978 (+ 6225us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 05:00:32.165989 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 05:00:32.165996 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 05:00:32.166045 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0806 05:00:32.169674 (+ 3629us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 05:00:32.171547 (+ 1873us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 05:00:32.171553 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 05:00:32.171557 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 05:00:32.171703 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 05:00:32.172233 (+ 530us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 05:00:32.172239 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 05:00:32.172242 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 05:00:32.172584 (+ 342us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 05:00:32.172712 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0806 05:00:32.173693 (+ 981us) server_negotiation.cc:299] Negotiation successful
0806 05:00:32.173956 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27956,"thread_start_us":30878,"threads_started":1}
I20260806 05:00:32.176533 29567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:00:32.125154 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:45703 (local address 127.9.96.66:37669)
0806 05:00:32.148108 (+ 22954us) negotiation.cc:107] Waiting for socket to connect
0806 05:00:32.148177 (+ 69us) client_negotiation.cc:174] Beginning negotiation
0806 05:00:32.151159 (+ 2982us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 05:00:32.152787 (+ 1628us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 05:00:32.152803 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 05:00:32.153545 (+ 742us) client_negotiation.cc:189] Negotiated authn=SASL
0806 05:00:32.154427 (+ 882us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 05:00:32.154460 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 05:00:32.156230 (+ 1770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 05:00:32.156237 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 05:00:32.158566 (+ 2329us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 05:00:32.158601 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 05:00:32.160155 (+ 1554us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 05:00:32.160176 (+ 21us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 05:00:32.160517 (+ 341us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:00:32.162481 (+ 1964us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 05:00:32.162520 (+ 39us) client_negotiation.cc:656] calling sasl_client_start()
0806 05:00:32.165698 (+ 3178us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 05:00:32.169845 (+ 4147us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 05:00:32.169855 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 05:00:32.169881 (+ 26us) client_negotiation.cc:760] calling sasl_client_step()
0806 05:00:32.171397 (+ 1516us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 05:00:32.171866 (+ 469us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 05:00:32.171871 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 05:00:32.171875 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 05:00:32.172096 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 05:00:32.172822 (+ 726us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 05:00:32.172831 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 05:00:32.173241 (+ 410us) client_negotiation.cc:769] Sending connection context
0806 05:00:32.174596 (+ 1355us) client_negotiation.cc:240] Negotiation successful
0806 05:00:32.175153 (+ 557us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":22604,"thread_start_us":22397,"threads_started":1}
I20260806 05:00:32.178289 29562 heartbeater.cc:347] Connected to a master server at 127.9.96.126:45703
I20260806 05:00:32.178628 29562 heartbeater.cc:464] Registering TS with master...
I20260806 05:00:32.179528 29562 heartbeater.cc:511] Master 127.9.96.126:45703 requested a full tablet report, sending...
I20260806 05:00:32.181643 29006 ts_manager.cc:194] Registered new tserver with Master: 6d361a41f2774ada80174965738071f4 (127.9.96.66:46109)
I20260806 05:00:32.184005 29006 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.96.66@KRBTEST.COM'} at 127.9.96.66:37669
I20260806 05:00:32.497085 29344 heartbeater.cc:464] Registering TS with master...
I20260806 05:00:32.497571 29344 heartbeater.cc:511] Master 127.9.96.126:45703 requested a full tablet report, sending...
I20260806 05:00:32.498940 29006 ts_manager.cc:194] Registered new tserver with Master: 871aee67d4564a37b4290d09c6257fc2 (127.9.96.65:44843)
I20260806 05:00:32.500447 29006 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.96.65@KRBTEST.COM'} at 127.9.96.65:57613
W20260806 05:00:32.564584 29569 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 05:00:32.564890 29569 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 05:00:32.565057 29569 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 05:00:32.573536 29569 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 05:00:32.573727 29569 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 05:00:32.573942 29569 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.96.67
I20260806 05:00:32.586776 29569 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.96.84:35379
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785992227050689-9601-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785992227050689-9601-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.9.96.84:43097/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.67.keytab
--principal=kudu/127.9.96.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.96.67:39039
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785992227050689-9601-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.96.67
--webserver_port=36413
--tserver_master_addrs=127.9.96.126:45703
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.96.67
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785992227050689-9601-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 03:56:12 UTC on ccycloud.quasar-guijbm.root.comops.site
build id 4569618
ASAN enabled
I20260806 05:00:32.588533 29569 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 05:00:32.589856 29569 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 05:00:32.609014 29578 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 05:00:32.610299 29569 server_base.cc:1034] running on GCE node
W20260806 05:00:32.610709 29585 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
Aug 06 05:00:32 dist-test-slave-x4qh krb5kdc[27396](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992432, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.96.126@KRBTEST.COM for HTTP/127.9.96.84@KRBTEST.COM
W20260806 05:00:32.611084 29579 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 05:00:32.611902 29569 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 05:00:32.613042 29569 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 05:00:32.614277 29569 hybrid_clock.cc:648] HybridClock initialized: now 1785992432614214 us; error 58 us; skew 500 ppm
I20260806 05:00:32.614661 29569 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 05:00:32 dist-test-slave-x4qh krb5kdc[27396](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992432, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.96.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 05:00:32.618855 29569 init.cc:377] Logged in from keytab as kudu/127.9.96.67@KRBTEST.COM (short username kudu)
I20260806 05:00:32.628479 29569 webserver.cc:460] Webserver started at http://127.9.96.67:36413/ using document root <none> and password file <none>
I20260806 05:00:32.629276 29569 fs_manager.cc:359] Metadata directory not provided
I20260806 05:00:32.629446 29569 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 06 05:00:32 dist-test-slave-x4qh krb5kdc[27396](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992432, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.96.67@KRBTEST.COM for HTTP/127.9.96.84@KRBTEST.COM
I20260806 05:00:32.665467 29569 fs_manager.cc:709] Time spent opening directory manager: real 0.019s user 0.011s sys 0.003s
I20260806 05:00:32.680773 29595 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 05:00:32.682770 29569 fs_manager.cc:725] Time spent opening block manager: real 0.015s user 0.005s sys 0.008s
I20260806 05:00:32.682978 29569 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785992227050689-9601-0/minicluster-data/ts-2/data,/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785992227050689-9601-0/minicluster-data/ts-2/wal
uuid: "0d27ed03047840a6bf076799cf5d5cd7"
format_stamp: "Formatted at 2026-08-06 05:00:29 on dist-test-slave-x4qh"
server_key: "fb2c155ce6b2e54dbb6a98ce8ed35fe3"
server_key_iv: "290ed9e2629c239ee383a90057f4c44b"
server_key_version: "kuduclusterkey@0"
I20260806 05:00:32.684520 29569 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785992227050689-9601-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785992227050689-9601-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785992227050689-9601-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 05:00:32.752878 29569 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 05:00:32.758188 29569 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 05:00:32.758505 29569 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 05:00:32.759831 29569 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 05:00:32.761286 29569 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 05:00:32.761381 29569 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 05:00:32.761531 29569 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 05:00:32.761628 29569 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 05:00:32.824004 29569 rpc_server.cc:307] RPC server started. Bound to: 127.9.96.67:39039
I20260806 05:00:32.824173 29778 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.96.67:39039 every 8 connection(s)
I20260806 05:00:32.825742 29569 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785992227050689-9601-0/minicluster-data/ts-2/data/info.pb
I20260806 05:00:32.830096 9601 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu as pid 29569
Aug 06 05:00:32 dist-test-slave-x4qh krb5kdc[27396](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992432, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.96.67@KRBTEST.COM for kudu/127.9.96.126@KRBTEST.COM
I20260806 05:00:32.885907 29784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:00:32.829268 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:45703 (local address 127.9.96.67:45031)
0806 05:00:32.859443 (+ 30175us) negotiation.cc:107] Waiting for socket to connect
0806 05:00:32.859514 (+ 71us) client_negotiation.cc:174] Beginning negotiation
0806 05:00:32.861854 (+ 2340us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 05:00:32.863020 (+ 1166us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 05:00:32.863032 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 05:00:32.863738 (+ 706us) client_negotiation.cc:189] Negotiated authn=SASL
0806 05:00:32.864607 (+ 869us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 05:00:32.864635 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 05:00:32.866209 (+ 1574us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 05:00:32.866216 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 05:00:32.869353 (+ 3137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 05:00:32.869391 (+ 38us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 05:00:32.870708 (+ 1317us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 05:00:32.870716 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 05:00:32.871572 (+ 856us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:00:32.872467 (+ 895us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 05:00:32.872498 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0806 05:00:32.875329 (+ 2831us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 05:00:32.881228 (+ 5899us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 05:00:32.881238 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 05:00:32.881257 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0806 05:00:32.881834 (+ 577us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 05:00:32.882256 (+ 422us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 05:00:32.882261 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 05:00:32.882264 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 05:00:32.882493 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 05:00:32.883213 (+ 720us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 05:00:32.883220 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 05:00:32.883660 (+ 440us) client_negotiation.cc:769] Sending connection context
0806 05:00:32.883966 (+ 306us) client_negotiation.cc:240] Negotiation successful
0806 05:00:32.884541 (+ 575us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":29829,"thread_start_us":29591,"threads_started":1}
I20260806 05:00:32.887151 29783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:00:32.828605 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.96.67:45031 (local address 127.9.96.126:45703)
0806 05:00:32.857283 (+ 28678us) server_negotiation.cc:206] Beginning negotiation
0806 05:00:32.857289 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 05:00:32.860043 (+ 2754us) server_negotiation.cc:407] Connection header received
0806 05:00:32.862334 (+ 2291us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 05:00:32.862340 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 05:00:32.862446 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 05:00:32.862551 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0806 05:00:32.864841 (+ 2290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:00:32.865991 (+ 1150us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:00:32.869546 (+ 3555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:00:32.870261 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:00:32.870384 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:00:32.876119 (+ 5735us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 05:00:32.876126 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 05:00:32.876131 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 05:00:32.876180 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0806 05:00:32.881028 (+ 4848us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 05:00:32.881974 (+ 946us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 05:00:32.881979 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 05:00:32.881982 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 05:00:32.882125 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 05:00:32.882626 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 05:00:32.882631 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 05:00:32.882634 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 05:00:32.883012 (+ 378us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 05:00:32.883141 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0806 05:00:32.886589 (+ 3448us) server_negotiation.cc:299] Negotiation successful
0806 05:00:32.886815 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28553,"thread_start_us":28685,"threads_started":1}
I20260806 05:00:32.888571 29779 heartbeater.cc:347] Connected to a master server at 127.9.96.126:45703
I20260806 05:00:32.888902 29779 heartbeater.cc:464] Registering TS with master...
I20260806 05:00:32.889241 29788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:00:32.857456 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46172 (local address 127.9.96.126:45703)
0806 05:00:32.866401 (+ 8945us) server_negotiation.cc:206] Beginning negotiation
0806 05:00:32.866406 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 05:00:32.866425 (+ 19us) server_negotiation.cc:407] Connection header received
0806 05:00:32.866472 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 05:00:32.866477 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 05:00:32.866564 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 05:00:32.866683 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0806 05:00:32.869596 (+ 2913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:00:32.870847 (+ 1251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:00:32.873876 (+ 3029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:00:32.874735 (+ 859us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:00:32.874866 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:00:32.879015 (+ 4149us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 05:00:32.879023 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 05:00:32.879032 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 05:00:32.879090 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0806 05:00:32.884159 (+ 5069us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 05:00:32.885115 (+ 956us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 05:00:32.885119 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 05:00:32.885122 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 05:00:32.885265 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 05:00:32.885773 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 05:00:32.885779 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 05:00:32.885782 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 05:00:32.886748 (+ 966us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 05:00:32.888344 (+ 1596us) server_negotiation.cc:1035] Waiting for connection context
0806 05:00:32.888515 (+ 171us) server_negotiation.cc:299] Negotiation successful
0806 05:00:32.888811 (+ 296us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8796,"thread_start_us":9355,"threads_started":1}
I20260806 05:00:32.889760 29779 heartbeater.cc:511] Master 127.9.96.126:45703 requested a full tablet report, sending...
I20260806 05:00:32.891666 29006 ts_manager.cc:194] Registered new tserver with Master: 0d27ed03047840a6bf076799cf5d5cd7 (127.9.96.67:39039)
I20260806 05:00:32.892861 29006 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.96.67@KRBTEST.COM'} at 127.9.96.67:45031
I20260806 05:00:32.895594 9601 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 05:00:32.918403 29783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:00:32.900329 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46188 (local address 127.9.96.126:45703)
0806 05:00:32.900542 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0806 05:00:32.900547 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 05:00:32.901076 (+ 529us) server_negotiation.cc:407] Connection header received
0806 05:00:32.901335 (+ 259us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 05:00:32.901340 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 05:00:32.901414 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 05:00:32.901508 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0806 05:00:32.903615 (+ 2107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:00:32.904655 (+ 1040us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:00:32.906514 (+ 1859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:00:32.907178 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:00:32.907313 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:00:32.910823 (+ 3510us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 05:00:32.910830 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 05:00:32.910835 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 05:00:32.910880 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0806 05:00:32.914568 (+ 3688us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 05:00:32.915453 (+ 885us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 05:00:32.915458 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 05:00:32.915462 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 05:00:32.915617 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 05:00:32.916062 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 05:00:32.916067 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 05:00:32.916070 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 05:00:32.916437 (+ 367us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 05:00:32.916669 (+ 232us) server_negotiation.cc:1035] Waiting for connection context
0806 05:00:32.917857 (+ 1188us) server_negotiation.cc:299] Negotiation successful
0806 05:00:32.918063 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260806 05:00:32.924898 29006 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46188:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 05:00:32.936259 29006 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
Aug 06 05:00:32 dist-test-slave-x4qh krb5kdc[27396](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992432, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.96.126@KRBTEST.COM for HTTP/127.9.96.84@KRBTEST.COM
I20260806 05:00:32.985689 29810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:00:32.958670 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.66:46109 (local address 127.0.0.1:49112)
0806 05:00:32.975833 (+ 17163us) negotiation.cc:107] Waiting for socket to connect
0806 05:00:32.975853 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0806 05:00:32.976072 (+ 219us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 05:00:32.977267 (+ 1195us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 05:00:32.977275 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 05:00:32.977306 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 05:00:32.977761 (+ 455us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 05:00:32.977781 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 05:00:32.979853 (+ 2072us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 05:00:32.979858 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 05:00:32.982436 (+ 2578us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 05:00:32.982456 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 05:00:32.984702 (+ 2246us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 05:00:32.984708 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 05:00:32.984912 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:00:32.984962 (+ 50us) client_negotiation.cc:769] Sending connection context
0806 05:00:32.985094 (+ 132us) client_negotiation.cc:240] Negotiation successful
0806 05:00:32.985300 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":17036,"thread_start_us":15835,"threads_started":1}
I20260806 05:00:32.986608 29815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:00:32.959202 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49112 (local address 127.9.96.66:46109)
0806 05:00:32.968610 (+ 9408us) server_negotiation.cc:206] Beginning negotiation
0806 05:00:32.968619 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 05:00:32.976312 (+ 7693us) server_negotiation.cc:407] Connection header received
0806 05:00:32.976434 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 05:00:32.976442 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 05:00:32.976723 (+ 281us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 05:00:32.976937 (+ 214us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 05:00:32.978018 (+ 1081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:00:32.979667 (+ 1649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:00:32.982641 (+ 2974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:00:32.984161 (+ 1520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:00:32.984306 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:00:32.984527 (+ 221us) server_negotiation.cc:1035] Waiting for connection context
0806 05:00:32.986075 (+ 1548us) server_negotiation.cc:299] Negotiation successful
0806 05:00:32.986275 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9236,"thread_start_us":9085,"threads_started":1}
I20260806 05:00:32.990065 29817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:00:32.959214 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59806 (local address 127.9.96.67:39039)
0806 05:00:32.967744 (+ 8530us) server_negotiation.cc:206] Beginning negotiation
0806 05:00:32.967754 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 05:00:32.975136 (+ 7382us) server_negotiation.cc:407] Connection header received
0806 05:00:32.976636 (+ 1500us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 05:00:32.976646 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 05:00:32.976948 (+ 302us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 05:00:32.977141 (+ 193us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 05:00:32.978969 (+ 1828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:00:32.981192 (+ 2223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:00:32.986475 (+ 5283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:00:32.988027 (+ 1552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:00:32.988128 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:00:32.988350 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0806 05:00:32.989452 (+ 1102us) server_negotiation.cc:299] Negotiation successful
0806 05:00:32.989631 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8342,"thread_start_us":8187,"threads_started":1}
I20260806 05:00:32.990751 29813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:00:32.958452 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.67:39039 (local address 127.0.0.1:59806)
0806 05:00:32.974961 (+ 16509us) negotiation.cc:107] Waiting for socket to connect
0806 05:00:32.975010 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0806 05:00:32.976382 (+ 1372us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 05:00:32.977607 (+ 1225us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 05:00:32.977615 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 05:00:32.977655 (+ 40us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 05:00:32.978162 (+ 507us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 05:00:32.978176 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 05:00:32.981455 (+ 3279us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 05:00:32.981464 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 05:00:32.983750 (+ 2286us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 05:00:32.983774 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 05:00:32.988590 (+ 4816us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 05:00:32.988596 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 05:00:32.988781 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:00:32.988815 (+ 34us) client_negotiation.cc:769] Sending connection context
0806 05:00:32.990283 (+ 1468us) client_negotiation.cc:240] Negotiation successful
0806 05:00:32.990430 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":16204,"thread_start_us":17115,"threads_started":1}
I20260806 05:00:32.991315 29465 tablet_service.cc:1467] Processing CreateTablet for tablet 33a0333d6e76425198a8ec6c5b8c3ce3 (DEFAULT_TABLE table=test-table [id=1f70207b3af441dab59000501948a0c6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 05:00:32.992782 29465 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 33a0333d6e76425198a8ec6c5b8c3ce3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 05:00:33.006029 29670 tablet_service.cc:1467] Processing CreateTablet for tablet 33a0333d6e76425198a8ec6c5b8c3ce3 (DEFAULT_TABLE table=test-table [id=1f70207b3af441dab59000501948a0c6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 05:00:33.008121 29670 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 33a0333d6e76425198a8ec6c5b8c3ce3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 05:00:33.008827 29813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:00:32.976114 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.65:44843 (local address 127.0.0.1:50280)
0806 05:00:32.991716 (+ 15602us) negotiation.cc:107] Waiting for socket to connect
0806 05:00:32.991744 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0806 05:00:32.991894 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 05:00:33.002029 (+ 10135us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 05:00:33.002033 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 05:00:33.002062 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 05:00:33.002245 (+ 183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 05:00:33.002255 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 05:00:33.004478 (+ 2223us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 05:00:33.004484 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 05:00:33.006247 (+ 1763us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 05:00:33.006262 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 05:00:33.008090 (+ 1828us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 05:00:33.008096 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 05:00:33.008298 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:00:33.008332 (+ 34us) client_negotiation.cc:769] Sending connection context
0806 05:00:33.008397 (+ 65us) client_negotiation.cc:240] Negotiation successful
0806 05:00:33.008527 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14749,"thread_start_us":23503,"threads_started":1}
I20260806 05:00:33.009936 29821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:00:32.976447 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50280 (local address 127.9.96.65:44843)
0806 05:00:33.000311 (+ 23864us) server_negotiation.cc:206] Beginning negotiation
0806 05:00:33.000320 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 05:00:33.000351 (+ 31us) server_negotiation.cc:407] Connection header received
0806 05:00:33.000458 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 05:00:33.000466 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 05:00:33.000702 (+ 236us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 05:00:33.002439 (+ 1737us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 05:00:33.002712 (+ 273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:00:33.004278 (+ 1566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:00:33.006449 (+ 2171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:00:33.007903 (+ 1454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:00:33.009110 (+ 1207us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:00:33.009350 (+ 240us) server_negotiation.cc:1035] Waiting for connection context
0806 05:00:33.009419 (+ 69us) server_negotiation.cc:299] Negotiation successful
0806 05:00:33.009576 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23663,"thread_start_us":23479,"threads_started":1}
I20260806 05:00:33.014519 29228 tablet_service.cc:1467] Processing CreateTablet for tablet 33a0333d6e76425198a8ec6c5b8c3ce3 (DEFAULT_TABLE table=test-table [id=1f70207b3af441dab59000501948a0c6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 05:00:33.016053 29228 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 33a0333d6e76425198a8ec6c5b8c3ce3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 05:00:33.017462 29823 tablet_bootstrap.cc:492] T 33a0333d6e76425198a8ec6c5b8c3ce3 P 6d361a41f2774ada80174965738071f4: Bootstrap starting.
I20260806 05:00:33.023164 29823 tablet_bootstrap.cc:654] T 33a0333d6e76425198a8ec6c5b8c3ce3 P 6d361a41f2774ada80174965738071f4: Neither blocks nor log segments found. Creating new log.
I20260806 05:00:33.024665 29823 log.cc:824] T 33a0333d6e76425198a8ec6c5b8c3ce3 P 6d361a41f2774ada80174965738071f4: Log is configured to *not* fsync() on all Append() calls
I20260806 05:00:33.034507 29823 tablet_bootstrap.cc:492] T 33a0333d6e76425198a8ec6c5b8c3ce3 P 6d361a41f2774ada80174965738071f4: No bootstrap required, opened a new log
I20260806 05:00:33.034897 29823 ts_tablet_manager.cc:1397] T 33a0333d6e76425198a8ec6c5b8c3ce3 P 6d361a41f2774ada80174965738071f4: Time spent bootstrapping tablet: real 0.018s user 0.009s sys 0.005s
I20260806 05:00:33.044219 29826 tablet_bootstrap.cc:492] T 33a0333d6e76425198a8ec6c5b8c3ce3 P 871aee67d4564a37b4290d09c6257fc2: Bootstrap starting.
I20260806 05:00:33.048205 29826 tablet_bootstrap.cc:654] T 33a0333d6e76425198a8ec6c5b8c3ce3 P 871aee67d4564a37b4290d09c6257fc2: Neither blocks nor log segments found. Creating new log.
I20260806 05:00:33.050843 29823 raft_consensus.cc:348] T 33a0333d6e76425198a8ec6c5b8c3ce3 P 6d361a41f2774ada80174965738071f4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d27ed03047840a6bf076799cf5d5cd7" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 39039 } } peers { permanent_uuid: "6d361a41f2774ada80174965738071f4" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 46109 } } peers { permanent_uuid: "871aee67d4564a37b4290d09c6257fc2" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 44843 } }
I20260806 05:00:33.053205 29826 log.cc:824] T 33a0333d6e76425198a8ec6c5b8c3ce3 P 871aee67d4564a37b4290d09c6257fc2: Log is configured to *not* fsync() on all Append() calls
I20260806 05:00:33.053517 29823 raft_consensus.cc:374] T 33a0333d6e76425198a8ec6c5b8c3ce3 P 6d361a41f2774ada80174965738071f4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 05:00:33.053704 29823 raft_consensus.cc:729] T 33a0333d6e76425198a8ec6c5b8c3ce3 P 6d361a41f2774ada80174965738071f4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6d361a41f2774ada80174965738071f4, State: Initialized, Role: FOLLOWER
I20260806 05:00:33.054731 29823 consensus_queue.cc:260] T 33a0333d6e76425198a8ec6c5b8c3ce3 P 6d361a41f2774ada80174965738071f4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d27ed03047840a6bf076799cf5d5cd7" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 39039 } } peers { permanent_uuid: "6d361a41f2774ada80174965738071f4" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 46109 } } peers { permanent_uuid: "871aee67d4564a37b4290d09c6257fc2" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 44843 } }
I20260806 05:00:33.064604 29823 ts_tablet_manager.cc:1428] T 33a0333d6e76425198a8ec6c5b8c3ce3 P 6d361a41f2774ada80174965738071f4: Time spent starting tablet: real 0.029s user 0.012s sys 0.004s
I20260806 05:00:33.065003 29562 heartbeater.cc:503] Master 127.9.96.126:45703 was elected leader, sending a full tablet report...
I20260806 05:00:33.068261 29828 tablet_bootstrap.cc:492] T 33a0333d6e76425198a8ec6c5b8c3ce3 P 0d27ed03047840a6bf076799cf5d5cd7: Bootstrap starting.
I20260806 05:00:33.071112 29826 tablet_bootstrap.cc:492] T 33a0333d6e76425198a8ec6c5b8c3ce3 P 871aee67d4564a37b4290d09c6257fc2: No bootstrap required, opened a new log
I20260806 05:00:33.071522 29826 ts_tablet_manager.cc:1397] T 33a0333d6e76425198a8ec6c5b8c3ce3 P 871aee67d4564a37b4290d09c6257fc2: Time spent bootstrapping tablet: real 0.028s user 0.007s sys 0.005s
I20260806 05:00:33.079118 29828 tablet_bootstrap.cc:654] T 33a0333d6e76425198a8ec6c5b8c3ce3 P 0d27ed03047840a6bf076799cf5d5cd7: Neither blocks nor log segments found. Creating new log.
I20260806 05:00:33.080610 29828 log.cc:824] T 33a0333d6e76425198a8ec6c5b8c3ce3 P 0d27ed03047840a6bf076799cf5d5cd7: Log is configured to *not* fsync() on all Append() calls
I20260806 05:00:33.082821 29826 raft_consensus.cc:348] T 33a0333d6e76425198a8ec6c5b8c3ce3 P 871aee67d4564a37b4290d09c6257fc2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d27ed03047840a6bf076799cf5d5cd7" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 39039 } } peers { permanent_uuid: "6d361a41f2774ada80174965738071f4" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 46109 } } peers { permanent_uuid: "871aee67d4564a37b4290d09c6257fc2" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 44843 } }
I20260806 05:00:33.083551 29826 raft_consensus.cc:374] T 33a0333d6e76425198a8ec6c5b8c3ce3 P 871aee67d4564a37b4290d09c6257fc2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 05:00:33.083739 29826 raft_consensus.cc:729] T 33a0333d6e76425198a8ec6c5b8c3ce3 P 871aee67d4564a37b4290d09c6257fc2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 871aee67d4564a37b4290d09c6257fc2, State: Initialized, Role: FOLLOWER
I20260806 05:00:33.084757 29826 consensus_queue.cc:260] T 33a0333d6e76425198a8ec6c5b8c3ce3 P 871aee67d4564a37b4290d09c6257fc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d27ed03047840a6bf076799cf5d5cd7" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 39039 } } peers { permanent_uuid: "6d361a41f2774ada80174965738071f4" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 46109 } } peers { permanent_uuid: "871aee67d4564a37b4290d09c6257fc2" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 44843 } }
I20260806 05:00:33.091171 29826 ts_tablet_manager.cc:1428] T 33a0333d6e76425198a8ec6c5b8c3ce3 P 871aee67d4564a37b4290d09c6257fc2: Time spent starting tablet: real 0.019s user 0.013s sys 0.003s
I20260806 05:00:33.091641 29344 heartbeater.cc:503] Master 127.9.96.126:45703 was elected leader, sending a full tablet report...
I20260806 05:00:33.093770 29828 tablet_bootstrap.cc:492] T 33a0333d6e76425198a8ec6c5b8c3ce3 P 0d27ed03047840a6bf076799cf5d5cd7: No bootstrap required, opened a new log
I20260806 05:00:33.094082 29828 ts_tablet_manager.cc:1397] T 33a0333d6e76425198a8ec6c5b8c3ce3 P 0d27ed03047840a6bf076799cf5d5cd7: Time spent bootstrapping tablet: real 0.026s user 0.007s sys 0.007s
I20260806 05:00:33.105180 29828 raft_consensus.cc:348] T 33a0333d6e76425198a8ec6c5b8c3ce3 P 0d27ed03047840a6bf076799cf5d5cd7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d27ed03047840a6bf076799cf5d5cd7" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 39039 } } peers { permanent_uuid: "6d361a41f2774ada80174965738071f4" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 46109 } } peers { permanent_uuid: "871aee67d4564a37b4290d09c6257fc2" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 44843 } }
I20260806 05:00:33.105679 29828 raft_consensus.cc:374] T 33a0333d6e76425198a8ec6c5b8c3ce3 P 0d27ed03047840a6bf076799cf5d5cd7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 05:00:33.105791 29828 raft_consensus.cc:729] T 33a0333d6e76425198a8ec6c5b8c3ce3 P 0d27ed03047840a6bf076799cf5d5cd7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d27ed03047840a6bf076799cf5d5cd7, State: Initialized, Role: FOLLOWER
I20260806 05:00:33.106549 29828 consensus_queue.cc:260] T 33a0333d6e76425198a8ec6c5b8c3ce3 P 0d27ed03047840a6bf076799cf5d5cd7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d27ed03047840a6bf076799cf5d5cd7" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 39039 } } peers { permanent_uuid: "6d361a41f2774ada80174965738071f4" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 46109 } } peers { permanent_uuid: "871aee67d4564a37b4290d09c6257fc2" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 44843 } }
I20260806 05:00:33.123116 29830 raft_consensus.cc:482] T 33a0333d6e76425198a8ec6c5b8c3ce3 P 6d361a41f2774ada80174965738071f4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 05:00:33.123644 29830 raft_consensus.cc:504] T 33a0333d6e76425198a8ec6c5b8c3ce3 P 6d361a41f2774ada80174965738071f4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d27ed03047840a6bf076799cf5d5cd7" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 39039 } } peers { permanent_uuid: "6d361a41f2774ada80174965738071f4" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 46109 } } peers { permanent_uuid: "871aee67d4564a37b4290d09c6257fc2" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 44843 } }
I20260806 05:00:33.126381 29830 leader_election.cc:288] T 33a0333d6e76425198a8ec6c5b8c3ce3 P 6d361a41f2774ada80174965738071f4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0d27ed03047840a6bf076799cf5d5cd7 (127.9.96.67:39039), 871aee67d4564a37b4290d09c6257fc2 (127.9.96.65:44843)
W20260806 05:00:33.126879 29564 tablet.cc:2369] T 33a0333d6e76425198a8ec6c5b8c3ce3 P 6d361a41f2774ada80174965738071f4: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 05:00:33.134346 29779 heartbeater.cc:503] Master 127.9.96.126:45703 was elected leader, sending a full tablet report...
I20260806 05:00:33.140820 29817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:00:33.128483 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.96.66:45687 (local address 127.9.96.67:39039)
0806 05:00:33.128713 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0806 05:00:33.128720 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 05:00:33.133102 (+ 4382us) server_negotiation.cc:407] Connection header received
0806 05:00:33.133440 (+ 338us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 05:00:33.133446 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 05:00:33.133518 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 05:00:33.133575 (+ 57us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 05:00:33.134535 (+ 960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:00:33.135752 (+ 1217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:00:33.138378 (+ 2626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:00:33.139559 (+ 1181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:00:33.139656 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:00:33.139761 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0806 05:00:33.140421 (+ 660us) server_negotiation.cc:299] Negotiation successful
0806 05:00:33.140554 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260806 05:00:33.141572 29836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:00:33.128370 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.67:39039 (local address 127.9.96.66:45687)
0806 05:00:33.132980 (+ 4610us) negotiation.cc:107] Waiting for socket to connect
0806 05:00:33.133001 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0806 05:00:33.133263 (+ 262us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 05:00:33.133953 (+ 690us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 05:00:33.133958 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 05:00:33.133984 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 05:00:33.134341 (+ 357us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 05:00:33.134351 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 05:00:33.136105 (+ 1754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 05:00:33.136111 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 05:00:33.138195 (+ 2084us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 05:00:33.138211 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 05:00:33.139969 (+ 1758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 05:00:33.139976 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 05:00:33.140208 (+ 232us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:00:33.140243 (+ 35us) client_negotiation.cc:769] Sending connection context
0806 05:00:33.141064 (+ 821us) client_negotiation.cc:240] Negotiation successful
0806 05:00:33.141220 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4474,"thread_start_us":4301,"threads_started":1}
I20260806 05:00:33.143121 29704 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "33a0333d6e76425198a8ec6c5b8c3ce3" candidate_uuid: "6d361a41f2774ada80174965738071f4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d27ed03047840a6bf076799cf5d5cd7" is_pre_election: true
I20260806 05:00:33.143774 29704 raft_consensus.cc:2454] T 33a0333d6e76425198a8ec6c5b8c3ce3 P 0d27ed03047840a6bf076799cf5d5cd7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6d361a41f2774ada80174965738071f4 in term 0.
I20260806 05:00:33.144584 29394 leader_election.cc:302] T 33a0333d6e76425198a8ec6c5b8c3ce3 P 6d361a41f2774ada80174965738071f4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d27ed03047840a6bf076799cf5d5cd7, 6d361a41f2774ada80174965738071f4; no voters:
I20260806 05:00:33.145020 29830 raft_consensus.cc:2781] T 33a0333d6e76425198a8ec6c5b8c3ce3 P 6d361a41f2774ada80174965738071f4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 05:00:33.145185 29830 raft_consensus.cc:482] T 33a0333d6e76425198a8ec6c5b8c3ce3 P 6d361a41f2774ada80174965738071f4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 05:00:33.145298 29830 raft_consensus.cc:3037] T 33a0333d6e76425198a8ec6c5b8c3ce3 P 6d361a41f2774ada80174965738071f4 [term 0 FOLLOWER]: Advancing to term 1
I20260806 05:00:33.147984 29828 ts_tablet_manager.cc:1428] T 33a0333d6e76425198a8ec6c5b8c3ce3 P 0d27ed03047840a6bf076799cf5d5cd7: Time spent starting tablet: real 0.054s user 0.011s sys 0.025s
I20260806 05:00:33.148643 29830 raft_consensus.cc:504] T 33a0333d6e76425198a8ec6c5b8c3ce3 P 6d361a41f2774ada80174965738071f4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d27ed03047840a6bf076799cf5d5cd7" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 39039 } } peers { permanent_uuid: "6d361a41f2774ada80174965738071f4" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 46109 } } peers { permanent_uuid: "871aee67d4564a37b4290d09c6257fc2" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 44843 } }
I20260806 05:00:33.149406 29830 leader_election.cc:288] T 33a0333d6e76425198a8ec6c5b8c3ce3 P 6d361a41f2774ada80174965738071f4 [CANDIDATE]: Term 1 election: Requested vote from peers 0d27ed03047840a6bf076799cf5d5cd7 (127.9.96.67:39039), 871aee67d4564a37b4290d09c6257fc2 (127.9.96.65:44843)
I20260806 05:00:33.154698 29704 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "33a0333d6e76425198a8ec6c5b8c3ce3" candidate_uuid: "6d361a41f2774ada80174965738071f4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d27ed03047840a6bf076799cf5d5cd7"
I20260806 05:00:33.154956 29704 raft_consensus.cc:3037] T 33a0333d6e76425198a8ec6c5b8c3ce3 P 0d27ed03047840a6bf076799cf5d5cd7 [term 0 FOLLOWER]: Advancing to term 1
I20260806 05:00:33.158502 29704 raft_consensus.cc:2454] T 33a0333d6e76425198a8ec6c5b8c3ce3 P 0d27ed03047840a6bf076799cf5d5cd7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6d361a41f2774ada80174965738071f4 in term 1.
I20260806 05:00:33.159080 29394 leader_election.cc:302] T 33a0333d6e76425198a8ec6c5b8c3ce3 P 6d361a41f2774ada80174965738071f4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d27ed03047840a6bf076799cf5d5cd7, 6d361a41f2774ada80174965738071f4; no voters:
I20260806 05:00:33.159511 29830 raft_consensus.cc:2781] T 33a0333d6e76425198a8ec6c5b8c3ce3 P 6d361a41f2774ada80174965738071f4 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 05:00:33.167246 29830 raft_consensus.cc:686] T 33a0333d6e76425198a8ec6c5b8c3ce3 P 6d361a41f2774ada80174965738071f4 [term 1 LEADER]: Becoming Leader. State: Replica: 6d361a41f2774ada80174965738071f4, State: Running, Role: LEADER
I20260806 05:00:33.167886 29830 consensus_queue.cc:237] T 33a0333d6e76425198a8ec6c5b8c3ce3 P 6d361a41f2774ada80174965738071f4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d27ed03047840a6bf076799cf5d5cd7" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 39039 } } peers { permanent_uuid: "6d361a41f2774ada80174965738071f4" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 46109 } } peers { permanent_uuid: "871aee67d4564a37b4290d09c6257fc2" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 44843 } }
I20260806 05:00:33.172775 29821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:00:33.133447 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.96.66:59681 (local address 127.9.96.65:44843)
0806 05:00:33.133839 (+ 392us) server_negotiation.cc:206] Beginning negotiation
0806 05:00:33.133844 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 05:00:33.141902 (+ 8058us) server_negotiation.cc:407] Connection header received
0806 05:00:33.161296 (+ 19394us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 05:00:33.161304 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 05:00:33.161409 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 05:00:33.161477 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 05:00:33.163240 (+ 1763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:00:33.164443 (+ 1203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:00:33.170305 (+ 5862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:00:33.171594 (+ 1289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:00:33.171712 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:00:33.171820 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0806 05:00:33.172380 (+ 560us) server_negotiation.cc:299] Negotiation successful
0806 05:00:33.172522 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":243}
I20260806 05:00:33.173483 29836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:00:33.133149 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.65:44843 (local address 127.9.96.66:59681)
0806 05:00:33.141804 (+ 8655us) negotiation.cc:107] Waiting for socket to connect
0806 05:00:33.141823 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0806 05:00:33.160905 (+ 19082us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 05:00:33.161723 (+ 818us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 05:00:33.161728 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 05:00:33.161756 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 05:00:33.161936 (+ 180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 05:00:33.161947 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 05:00:33.164676 (+ 2729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 05:00:33.164682 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 05:00:33.166623 (+ 1941us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 05:00:33.166639 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 05:00:33.171968 (+ 5329us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 05:00:33.171974 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 05:00:33.172182 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:00:33.172216 (+ 34us) client_negotiation.cc:769] Sending connection context
0806 05:00:33.173037 (+ 821us) client_negotiation.cc:240] Negotiation successful
0806 05:00:33.173208 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8546,"thread_start_us":8901,"threads_started":1}
I20260806 05:00:33.174459 29262 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "33a0333d6e76425198a8ec6c5b8c3ce3" candidate_uuid: "6d361a41f2774ada80174965738071f4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "871aee67d4564a37b4290d09c6257fc2" is_pre_election: true
I20260806 05:00:33.175051 29262 raft_consensus.cc:2454] T 33a0333d6e76425198a8ec6c5b8c3ce3 P 871aee67d4564a37b4290d09c6257fc2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6d361a41f2774ada80174965738071f4 in term 0.
I20260806 05:00:33.175944 29262 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "33a0333d6e76425198a8ec6c5b8c3ce3" candidate_uuid: "6d361a41f2774ada80174965738071f4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "871aee67d4564a37b4290d09c6257fc2"
I20260806 05:00:33.176136 29262 raft_consensus.cc:3037] T 33a0333d6e76425198a8ec6c5b8c3ce3 P 871aee67d4564a37b4290d09c6257fc2 [term 0 FOLLOWER]: Advancing to term 1
I20260806 05:00:33.179979 29262 raft_consensus.cc:2454] T 33a0333d6e76425198a8ec6c5b8c3ce3 P 871aee67d4564a37b4290d09c6257fc2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6d361a41f2774ada80174965738071f4 in term 1.
I20260806 05:00:33.183723 29006 catalog_manager.cc:5495] T 33a0333d6e76425198a8ec6c5b8c3ce3 P 6d361a41f2774ada80174965738071f4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6d361a41f2774ada80174965738071f4 (127.9.96.66). New cstate: current_term: 1 leader_uuid: "6d361a41f2774ada80174965738071f4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d27ed03047840a6bf076799cf5d5cd7" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 39039 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6d361a41f2774ada80174965738071f4" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 46109 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "871aee67d4564a37b4290d09c6257fc2" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 44843 } health_report { overall_health: UNKNOWN } } }
W20260806 05:00:33.203320 29345 tablet.cc:2369] T 33a0333d6e76425198a8ec6c5b8c3ce3 P 871aee67d4564a37b4290d09c6257fc2: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 05:00:33.226310 29815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:00:33.218233 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49120 (local address 127.9.96.66:46109)
0806 05:00:33.218417 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0806 05:00:33.218422 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 05:00:33.218443 (+ 21us) server_negotiation.cc:407] Connection header received
0806 05:00:33.218584 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 05:00:33.218590 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 05:00:33.218716 (+ 126us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 05:00:33.218900 (+ 184us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 05:00:33.219366 (+ 466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:00:33.220621 (+ 1255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:00:33.222849 (+ 2228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:00:33.223502 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:00:33.223639 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:00:33.224405 (+ 766us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 05:00:33.225121 (+ 716us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 05:00:33.225266 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0806 05:00:33.225926 (+ 660us) server_negotiation.cc:299] Negotiation successful
0806 05:00:33.226012 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260806 05:00:33.240775 29262 raft_consensus.cc:1261] T 33a0333d6e76425198a8ec6c5b8c3ce3 P 871aee67d4564a37b4290d09c6257fc2 [term 1 FOLLOWER]: Refusing update from remote peer 6d361a41f2774ada80174965738071f4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 05:00:33.240864 29704 raft_consensus.cc:1261] T 33a0333d6e76425198a8ec6c5b8c3ce3 P 0d27ed03047840a6bf076799cf5d5cd7 [term 1 FOLLOWER]: Refusing update from remote peer 6d361a41f2774ada80174965738071f4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 05:00:33.242029 29830 consensus_queue.cc:1035] T 33a0333d6e76425198a8ec6c5b8c3ce3 P 6d361a41f2774ada80174965738071f4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0d27ed03047840a6bf076799cf5d5cd7" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 39039 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 05:00:33.242452 29840 consensus_queue.cc:1035] T 33a0333d6e76425198a8ec6c5b8c3ce3 P 6d361a41f2774ada80174965738071f4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "871aee67d4564a37b4290d09c6257fc2" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 44843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 05:00:33.262471 29851 mvcc.cc:205] Tried to move back new op lower bound from 7315425006542868480 to 7315425006260760576. Current Snapshot: MvccSnapshot[applied={T|T < 7315425006542868480}]
I20260806 05:00:33.263542 29852 mvcc.cc:205] Tried to move back new op lower bound from 7315425006542868480 to 7315425006260760576. Current Snapshot: MvccSnapshot[applied={T|T < 7315425006542868480}]
I20260806 05:00:33.283026 29853 mvcc.cc:205] Tried to move back new op lower bound from 7315425006542868480 to 7315425006260760576. Current Snapshot: MvccSnapshot[applied={T|T < 7315425006542868480}]
I20260806 05:00:33.307174 29006 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46188:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 05:00:33.307514 29006 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46188:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 05:00:33.312189 29006 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 57daa83eda8242b5af95a57caa6b61a8: Sending DeleteTablet for 3 replicas of tablet 33a0333d6e76425198a8ec6c5b8c3ce3
I20260806 05:00:33.313508 29465 tablet_service.cc:1514] Processing DeleteTablet for tablet 33a0333d6e76425198a8ec6c5b8c3ce3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 05:00:33 UTC) from {username='kudu', principal='kudu/127.9.96.126@KRBTEST.COM'} at 127.0.0.1:49112
I20260806 05:00:33.313964 29670 tablet_service.cc:1514] Processing DeleteTablet for tablet 33a0333d6e76425198a8ec6c5b8c3ce3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 05:00:33 UTC) from {username='kudu', principal='kudu/127.9.96.126@KRBTEST.COM'} at 127.0.0.1:59806
I20260806 05:00:33.315248 29228 tablet_service.cc:1514] Processing DeleteTablet for tablet 33a0333d6e76425198a8ec6c5b8c3ce3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 05:00:33 UTC) from {username='kudu', principal='kudu/127.9.96.126@KRBTEST.COM'} at 127.0.0.1:50280
I20260806 05:00:33.322734 29865 tablet_replica.cc:330] T 33a0333d6e76425198a8ec6c5b8c3ce3 P 6d361a41f2774ada80174965738071f4: stopping tablet replica
I20260806 05:00:33.323325 29870 tablet_replica.cc:330] T 33a0333d6e76425198a8ec6c5b8c3ce3 P 871aee67d4564a37b4290d09c6257fc2: stopping tablet replica
I20260806 05:00:33.323926 29869 tablet_replica.cc:330] T 33a0333d6e76425198a8ec6c5b8c3ce3 P 0d27ed03047840a6bf076799cf5d5cd7: stopping tablet replica
I20260806 05:00:33.324496 29865 raft_consensus.cc:2229] T 33a0333d6e76425198a8ec6c5b8c3ce3 P 6d361a41f2774ada80174965738071f4 [term 1 LEADER]: Raft consensus shutting down.
I20260806 05:00:33.325646 29870 raft_consensus.cc:2229] T 33a0333d6e76425198a8ec6c5b8c3ce3 P 871aee67d4564a37b4290d09c6257fc2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 05:00:33.326730 29865 raft_consensus.cc:2258] T 33a0333d6e76425198a8ec6c5b8c3ce3 P 6d361a41f2774ada80174965738071f4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 05:00:33.326944 29870 raft_consensus.cc:2258] T 33a0333d6e76425198a8ec6c5b8c3ce3 P 871aee67d4564a37b4290d09c6257fc2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 05:00:33.327433 29869 raft_consensus.cc:2229] T 33a0333d6e76425198a8ec6c5b8c3ce3 P 0d27ed03047840a6bf076799cf5d5cd7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 05:00:33.327924 29869 raft_consensus.cc:2258] T 33a0333d6e76425198a8ec6c5b8c3ce3 P 0d27ed03047840a6bf076799cf5d5cd7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 05:00:33.329041 9601 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu pid 29075
I20260806 05:00:33.329473 29865 ts_tablet_manager.cc:1905] T 33a0333d6e76425198a8ec6c5b8c3ce3 P 6d361a41f2774ada80174965738071f4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 05:00:33.339640 29869 ts_tablet_manager.cc:1905] T 33a0333d6e76425198a8ec6c5b8c3ce3 P 0d27ed03047840a6bf076799cf5d5cd7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 05:00:33.341349 29870 ts_tablet_manager.cc:1905] T 33a0333d6e76425198a8ec6c5b8c3ce3 P 871aee67d4564a37b4290d09c6257fc2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 05:00:33.347530 29821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:00:33.330689 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50292 (local address 127.9.96.65:44843)
0806 05:00:33.330857 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0806 05:00:33.330863 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 05:00:33.330883 (+ 20us) server_negotiation.cc:407] Connection header received
0806 05:00:33.330945 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 05:00:33.330950 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 05:00:33.331044 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 05:00:33.331154 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0806 05:00:33.333223 (+ 2069us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:00:33.334380 (+ 1157us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:00:33.336253 (+ 1873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:00:33.336887 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:00:33.337027 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:00:33.340428 (+ 3401us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 05:00:33.340443 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 05:00:33.340461 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 05:00:33.340501 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0806 05:00:33.344354 (+ 3853us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 05:00:33.345235 (+ 881us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 05:00:33.345244 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 05:00:33.345248 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 05:00:33.345412 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 05:00:33.345835 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 05:00:33.345841 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 05:00:33.345844 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 05:00:33.346272 (+ 428us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 05:00:33.346598 (+ 326us) server_negotiation.cc:1035] Waiting for connection context
0806 05:00:33.346938 (+ 340us) server_negotiation.cc:299] Negotiation successful
0806 05:00:33.347151 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260806 05:00:33.348194 29336 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 05:00:33.348311 29865 ts_tablet_manager.cc:1918] T 33a0333d6e76425198a8ec6c5b8c3ce3 P 6d361a41f2774ada80174965738071f4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 05:00:33.348554 29865 log.cc:1196] T 33a0333d6e76425198a8ec6c5b8c3ce3 P 6d361a41f2774ada80174965738071f4: Deleting WAL directory at /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785992227050689-9601-0/minicluster-data/ts-1/wal/wals/33a0333d6e76425198a8ec6c5b8c3ce3
I20260806 05:00:33.349231 29865 ts_tablet_manager.cc:1939] T 33a0333d6e76425198a8ec6c5b8c3ce3 P 6d361a41f2774ada80174965738071f4: Deleting consensus metadata
I20260806 05:00:33.351203 28982 catalog_manager.cc:4841] TS 6d361a41f2774ada80174965738071f4 (127.9.96.66:46109): tablet 33a0333d6e76425198a8ec6c5b8c3ce3 (table test-table [id=1f70207b3af441dab59000501948a0c6]) successfully deleted
I20260806 05:00:33.358078 29869 ts_tablet_manager.cc:1918] T 33a0333d6e76425198a8ec6c5b8c3ce3 P 0d27ed03047840a6bf076799cf5d5cd7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 05:00:33.360203 29869 log.cc:1196] T 33a0333d6e76425198a8ec6c5b8c3ce3 P 0d27ed03047840a6bf076799cf5d5cd7: Deleting WAL directory at /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785992227050689-9601-0/minicluster-data/ts-2/wal/wals/33a0333d6e76425198a8ec6c5b8c3ce3
I20260806 05:00:33.361049 29869 ts_tablet_manager.cc:1939] T 33a0333d6e76425198a8ec6c5b8c3ce3 P 0d27ed03047840a6bf076799cf5d5cd7: Deleting consensus metadata
I20260806 05:00:33.363066 28984 catalog_manager.cc:4841] TS 0d27ed03047840a6bf076799cf5d5cd7 (127.9.96.67:39039): tablet 33a0333d6e76425198a8ec6c5b8c3ce3 (table test-table [id=1f70207b3af441dab59000501948a0c6]) successfully deleted
I20260806 05:00:33.522063 9601 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu with pid 29075
W20260806 05:00:33.560274 28984 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.9.96.65:44843 (error 108)
I20260806 05:00:33.561139 9601 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu pid 29349
I20260806 05:00:33.580900 29815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:00:33.562688 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49126 (local address 127.9.96.66:46109)
0806 05:00:33.562890 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0806 05:00:33.562896 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 05:00:33.562914 (+ 18us) server_negotiation.cc:407] Connection header received
0806 05:00:33.562983 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 05:00:33.562988 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 05:00:33.563067 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 05:00:33.563177 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0806 05:00:33.565173 (+ 1996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:00:33.566473 (+ 1300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:00:33.568421 (+ 1948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:00:33.569140 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:00:33.569265 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:00:33.572768 (+ 3503us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 05:00:33.572783 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 05:00:33.572809 (+ 26us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 05:00:33.572863 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0806 05:00:33.577222 (+ 4359us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 05:00:33.578114 (+ 892us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 05:00:33.578123 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 05:00:33.578127 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 05:00:33.578295 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 05:00:33.578707 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 05:00:33.578711 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 05:00:33.578714 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 05:00:33.579131 (+ 417us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 05:00:33.580082 (+ 951us) server_negotiation.cc:1035] Waiting for connection context
0806 05:00:33.580264 (+ 182us) server_negotiation.cc:299] Negotiation successful
0806 05:00:33.580494 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260806 05:00:33.582508 29555 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 05:00:33.582986 29884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:00:33.576224 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.65:44843 (local address 127.0.0.1:50308)
0806 05:00:33.582642 (+ 6418us) negotiation.cc:107] Waiting for socket to connect
0806 05:00:33.582805 (+ 163us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.96.65:44843: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":6215,"thread_start_us":6089,"threads_started":1}
W20260806 05:00:33.583464 28984 catalog_manager.cc:4584] TS 871aee67d4564a37b4290d09c6257fc2 (127.9.96.65:44843): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 33a0333d6e76425198a8ec6c5b8c3ce3: Network error: Client connection negotiation failed: client connection to 127.9.96.65:44843: connect: Connection refused (error 111)
I20260806 05:00:33.637321 29884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:00:33.636714 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.65:44843 (local address 127.0.0.1:50310)
0806 05:00:33.637007 (+ 293us) negotiation.cc:107] Waiting for socket to connect
0806 05:00:33.637116 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.96.65:44843: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":84}
I20260806 05:00:33.638520 29884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:00:33.638083 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.65:44843 (local address 127.0.0.1:50318)
0806 05:00:33.638247 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0806 05:00:33.638351 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.96.65:44843: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260806 05:00:33.697540 9601 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu with pid 29349
I20260806 05:00:33.701225 29884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:00:33.700552 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.65:44843 (local address 127.0.0.1:50326)
0806 05:00:33.700887 (+ 335us) negotiation.cc:107] Waiting for socket to connect
0806 05:00:33.701009 (+ 122us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.96.65:44843: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":175}
I20260806 05:00:33.702548 29884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:00:33.702094 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.65:44843 (local address 127.0.0.1:50334)
0806 05:00:33.702335 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0806 05:00:33.702418 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.96.65:44843: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118}
I20260806 05:00:33.723631 9601 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu pid 29569
I20260806 05:00:33.738981 29887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:00:33.724509 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59808 (local address 127.9.96.67:39039)
0806 05:00:33.725038 (+ 529us) server_negotiation.cc:206] Beginning negotiation
0806 05:00:33.725043 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 05:00:33.725059 (+ 16us) server_negotiation.cc:407] Connection header received
0806 05:00:33.725116 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 05:00:33.725119 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 05:00:33.725177 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 05:00:33.725272 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0806 05:00:33.726903 (+ 1631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:00:33.727717 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:00:33.729567 (+ 1850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:00:33.729960 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:00:33.730068 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:00:33.733287 (+ 3219us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 05:00:33.733301 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 05:00:33.733318 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 05:00:33.733354 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0806 05:00:33.736521 (+ 3167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 05:00:33.737172 (+ 651us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 05:00:33.737180 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 05:00:33.737182 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 05:00:33.737333 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 05:00:33.737659 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 05:00:33.737662 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 05:00:33.737664 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 05:00:33.737989 (+ 325us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 05:00:33.738238 (+ 249us) server_negotiation.cc:1035] Waiting for connection context
0806 05:00:33.738425 (+ 187us) server_negotiation.cc:299] Negotiation successful
0806 05:00:33.738586 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":433,"thread_start_us":305,"threads_started":1}
I20260806 05:00:33.739681 29771 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 05:00:33.783591 29884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:00:33.783090 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.65:44843 (local address 127.0.0.1:50346)
0806 05:00:33.783305 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0806 05:00:33.783447 (+ 142us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.96.65:44843: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
I20260806 05:00:33.784722 29884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:00:33.784298 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.65:44843 (local address 127.0.0.1:50354)
0806 05:00:33.784517 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0806 05:00:33.784594 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.96.65:44843: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102}
I20260806 05:00:33.830541 9601 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu with pid 29569
I20260806 05:00:33.854365 9601 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu pid 28950
I20260806 05:00:33.855640 29059 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 05:00:33.967335 29884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:00:33.966846 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.65:44843 (local address 127.0.0.1:50368)
0806 05:00:33.967040 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0806 05:00:33.967136 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.96.65:44843: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
I20260806 05:00:33.967352 9601 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu with pid 28950
I20260806 05:00:33.997722 9601 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260806 05:00:34.343259 9601 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260806 05:00:34.343582 9601 mini_ranger.cc:66] Stopping Ranger...
I20260806 05:00:34.699453 9601 mini_ranger.cc:68] Stopped Ranger
2026-08-06 05:00:34.699 UTC [27448] LOG: received smart shutdown request
2026-08-06 05:00:34.702 UTC [27448] LOG: background worker "logical replication launcher" (PID 27468) exited with exit code 1
2026-08-06 05:00:34.702 UTC [27456] LOG: shutting down
2026-08-06 05:00:34.854 UTC [27448] LOG: database system is shut down
2026-08-06T05:00:34Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (53800 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785992227050689-9601-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 05:00:35 dist-test-slave-x4qh krb5kdc[29902](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 05:00:35 dist-test-slave-x4qh krb5kdc[29902](info): set up 2 sockets
Aug 06 05:00:35 dist-test-slave-x4qh krb5kdc[29902](info): commencing operation
krb5kdc: starting...
W20260806 05:00:36.297097 9601 mini_kdc.cc:121] Time spent starting KDC: real 1.267s user 0.001s sys 0.030s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785992227050689-9601-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785992227050689-9601-0/krb5kdc/test-user.keytab.
Aug 06 05:00:36 dist-test-slave-x4qh krb5kdc[29902](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992436, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T05:00:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T05:00:36Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.96.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.96.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.96.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.126.keytab.
Entry for principal kudu/127.9.96.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.126.keytab.
I20260806 05:00:36.550786 9601 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
/tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785992227050689-9601-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785992227050689-9601-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785992227050689-9601-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785992227050689-9601-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.96.126:36525
--webserver_interface=127.9.96.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.126.keytab
--principal=kudu/127.9.96.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.96.84:40965
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.96.126:36525,127.9.96.125:43181,127.9.96.124:37647
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785992227050689-9601-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785992227050689-9601-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785992227050689-9601-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 05:00:36.896930 29920 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 05:00:36.897243 29920 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 05:00:36.897333 29920 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 05:00:36.904520 29920 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 05:00:36.904620 29920 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 05:00:36.904675 29920 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 05:00:36.904742 29920 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 05:00:36.904786 29920 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 05:00:36.914634 29920 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.96.84:40965
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785992227050689-9601-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785992227050689-9601-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.96.126:36525,127.9.96.125:43181,127.9.96.124:37647
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.126.keytab
--principal=kudu/127.9.96.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.96.126:36525
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785992227050689-9601-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.96.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785992227050689-9601-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 03:56:12 UTC on ccycloud.quasar-guijbm.root.comops.site
build id 4569618
ASAN enabled
I20260806 05:00:36.915915 29920 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 05:00:36.917244 29920 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 05:00:36.928992 29931 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 05:00:36.929042 29935 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 05:00:36.929064 29932 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 05:00:36.929905 29920 server_base.cc:1034] running on GCE node
I20260806 05:00:36.930687 29920 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 05:00:36.932049 29920 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 05:00:36.933286 29920 hybrid_clock.cc:648] HybridClock initialized: now 1785992436933214 us; error 53 us; skew 500 ppm
I20260806 05:00:36.933663 29920 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 05:00:36 dist-test-slave-x4qh krb5kdc[29902](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992436, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.96.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 05:00:36.936796 29920 init.cc:377] Logged in from keytab as kudu/127.9.96.126@KRBTEST.COM (short username kudu)
I20260806 05:00:36.938215 29920 webserver.cc:460] Webserver started at http://127.9.96.126:41565/ using document root <none> and password file <none>
I20260806 05:00:36.938907 29920 fs_manager.cc:359] Metadata directory not provided
I20260806 05:00:36.939054 29920 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 05:00:36.939410 29920 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 05:00:36.942380 29920 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785992227050689-9601-0/minicluster-data/master-0/data/instance:
uuid: "dd7bd28b7a2e47c48fdbf033821161af"
format_stamp: "Formatted at 2026-08-06 05:00:36 on dist-test-slave-x4qh"
server_key: "0ec2e83d5f766577cd0a6ffd37a04ebd"
server_key_iv: "3e6722f815e6cba74d489d16c6b27fbf"
server_key_version: "encryptionkey@0"
I20260806 05:00:36.943210 29920 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785992227050689-9601-0/minicluster-data/master-0/wal/instance:
uuid: "dd7bd28b7a2e47c48fdbf033821161af"
format_stamp: "Formatted at 2026-08-06 05:00:36 on dist-test-slave-x4qh"
server_key: "0ec2e83d5f766577cd0a6ffd37a04ebd"
server_key_iv: "3e6722f815e6cba74d489d16c6b27fbf"
server_key_version: "encryptionkey@0"
I20260806 05:00:36.948443 29920 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260806 05:00:36.951915 29944 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 05:00:36.953234 29920 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260806 05:00:36.953440 29920 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785992227050689-9601-0/minicluster-data/master-0/data,/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785992227050689-9601-0/minicluster-data/master-0/wal
uuid: "dd7bd28b7a2e47c48fdbf033821161af"
format_stamp: "Formatted at 2026-08-06 05:00:36 on dist-test-slave-x4qh"
server_key: "0ec2e83d5f766577cd0a6ffd37a04ebd"
server_key_iv: "3e6722f815e6cba74d489d16c6b27fbf"
server_key_version: "encryptionkey@0"
I20260806 05:00:36.953642 29920 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785992227050689-9601-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785992227050689-9601-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785992227050689-9601-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 05:00:36.981056 29920 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 05:00:36.985062 29920 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 05:00:36.985373 29920 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 05:00:37.004544 29920 rpc_server.cc:307] RPC server started. Bound to: 127.9.96.126:36525
I20260806 05:00:37.004632 30027 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.96.126:36525 every 8 connection(s)
I20260806 05:00:37.005962 29920 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785992227050689-9601-0/minicluster-data/master-0/data/info.pb
I20260806 05:00:37.010049 30028 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 05:00:37.014962 9601 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu as pid 29920
I20260806 05:00:37.015192 9601 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785992227050689-9601-0/minicluster-data/master-0/wal/instance
I20260806 05:00:37.015964 9601 external_mini_cluster.cc:1428] Setting key 24e8c217755c4f5de72045d71d8a6497
I20260806 05:00:37.016921 30028 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.96.126" port: 36525 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 05:00:37 dist-test-slave-x4qh krb5kdc[29902](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992436, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.96.126@KRBTEST.COM for kudu/127.9.96.126@KRBTEST.COM
I20260806 05:00:37.039407 30031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:00:37.020224 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:36525 (local address 127.0.0.1:43358)
0806 05:00:37.021420 (+ 1196us) negotiation.cc:107] Waiting for socket to connect
0806 05:00:37.021453 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0806 05:00:37.023168 (+ 1715us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 05:00:37.024008 (+ 840us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 05:00:37.024015 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 05:00:37.024736 (+ 721us) client_negotiation.cc:189] Negotiated authn=SASL
0806 05:00:37.025076 (+ 340us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 05:00:37.025093 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 05:00:37.026642 (+ 1549us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 05:00:37.026649 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 05:00:37.027774 (+ 1125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 05:00:37.027787 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 05:00:37.028715 (+ 928us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 05:00:37.028722 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 05:00:37.028975 (+ 253us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:00:37.029702 (+ 727us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 05:00:37.029725 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0806 05:00:37.031963 (+ 2238us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 05:00:37.035142 (+ 3179us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 05:00:37.035155 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 05:00:37.035167 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0806 05:00:37.035635 (+ 468us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 05:00:37.036051 (+ 416us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 05:00:37.036057 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 05:00:37.036063 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0806 05:00:37.036224 (+ 161us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 05:00:37.036983 (+ 759us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 05:00:37.036990 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 05:00:37.037218 (+ 228us) client_negotiation.cc:769] Sending connection context
0806 05:00:37.037452 (+ 234us) client_negotiation.cc:240] Negotiation successful
0806 05:00:37.038024 (+ 572us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":997,"mutex_wait_us":189,"thread_start_us":772,"threads_started":1}
I20260806 05:00:37.039444 30030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:00:37.020223 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43358 (local address 127.9.96.126:36525)
0806 05:00:37.021139 (+ 916us) server_negotiation.cc:206] Beginning negotiation
0806 05:00:37.021159 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0806 05:00:37.021585 (+ 426us) server_negotiation.cc:407] Connection header received
0806 05:00:37.023552 (+ 1967us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 05:00:37.023558 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 05:00:37.023768 (+ 210us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 05:00:37.023976 (+ 208us) server_negotiation.cc:226] Negotiated authn=SASL
0806 05:00:37.025280 (+ 1304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:00:37.026487 (+ 1207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:00:37.027922 (+ 1435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:00:37.028471 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:00:37.028667 (+ 196us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:00:37.032184 (+ 3517us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 05:00:37.032192 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 05:00:37.032205 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 05:00:37.032233 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0806 05:00:37.035014 (+ 2781us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 05:00:37.035770 (+ 756us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 05:00:37.035783 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 05:00:37.035785 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 05:00:37.035939 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 05:00:37.036342 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 05:00:37.036351 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 05:00:37.036354 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 05:00:37.036847 (+ 493us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 05:00:37.037047 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0806 05:00:37.037573 (+ 526us) server_negotiation.cc:299] Negotiation successful
0806 05:00:37.037873 (+ 300us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":639,"thread_start_us":412,"threads_started":1}
WARNING: no policy specified for kudu/127.9.96.125@KRBTEST.COM; defaulting to no policy
I20260806 05:00:37.042390 30028 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.96.125" port: 43181 } has no permanent_uuid. Determining permanent_uuid...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.96.125@KRBTEST.COM" created.
I20260806 05:00:37.043360 30031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:00:37.042979 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.125:43181 (local address 127.0.0.1:45450)
0806 05:00:37.043144 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0806 05:00:37.043249 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.96.125:43181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
W20260806 05:00:37.043651 29950 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.9.96.125:43181: connect: Connection refused (error 111)
I20260806 05:00:37.045173 30031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:00:37.044830 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.125:43181 (local address 127.0.0.1:45458)
0806 05:00:37.044989 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0806 05:00:37.045051 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.96.125:43181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
W20260806 05:00:37.045392 30028 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.96.125:43181: Network error: Client connection negotiation failed: client connection to 127.9.96.125:43181: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.96.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.125.keytab.
Entry for principal kudu/127.9.96.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.125.keytab.
I20260806 05:00:37.074668 9601 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
/tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785992227050689-9601-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785992227050689-9601-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785992227050689-9601-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785992227050689-9601-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.96.125:43181
--webserver_interface=127.9.96.125
--webserver_port=0
--keytab_file=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.125.keytab
--principal=kudu/127.9.96.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.96.84:40965
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.96.126:36525,127.9.96.125:43181,127.9.96.124:37647
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785992227050689-9601-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785992227050689-9601-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785992227050689-9601-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 05:00:37.094753 30028 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.96.125" port: 43181 } attempt: 1
I20260806 05:00:37.096210 30031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:00:37.095629 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.125:43181 (local address 127.0.0.1:45468)
0806 05:00:37.095842 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0806 05:00:37.095984 (+ 142us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.96.125:43181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
I20260806 05:00:37.097440 30031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:00:37.097011 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.125:43181 (local address 127.0.0.1:45478)
0806 05:00:37.097201 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0806 05:00:37.097303 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.96.125:43181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
W20260806 05:00:37.097635 30028 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.96.125:43181: Network error: Client connection negotiation failed: client connection to 127.9.96.125:43181: connect: Connection refused (error 111)
I20260806 05:00:37.165969 30028 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.96.125" port: 43181 } attempt: 2
I20260806 05:00:37.167145 30031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:00:37.166643 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.125:43181 (local address 127.0.0.1:45494)
0806 05:00:37.166909 (+ 266us) negotiation.cc:107] Waiting for socket to connect
0806 05:00:37.167002 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.96.125:43181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74}
I20260806 05:00:37.168134 30031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:00:37.167780 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.125:43181 (local address 127.0.0.1:45496)
0806 05:00:37.167943 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0806 05:00:37.168006 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.96.125:43181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260806 05:00:37.168308 30028 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.96.125:43181: Network error: Client connection negotiation failed: client connection to 127.9.96.125:43181: connect: Connection refused (error 111)
I20260806 05:00:37.259569 30028 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.96.125" port: 43181 } attempt: 3
I20260806 05:00:37.260761 30031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:00:37.260277 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.125:43181 (local address 127.0.0.1:45512)
0806 05:00:37.260486 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0806 05:00:37.260609 (+ 123us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.96.125:43181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":76}
I20260806 05:00:37.261776 30031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:00:37.261414 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.125:43181 (local address 127.0.0.1:45526)
0806 05:00:37.261565 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0806 05:00:37.261635 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.96.125:43181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260806 05:00:37.261943 30028 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.96.125:43181: Network error: Client connection negotiation failed: client connection to 127.9.96.125:43181: connect: Connection refused (error 111)
W20260806 05:00:37.404364 30037 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 05:00:37.404742 30037 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 05:00:37.404906 30037 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260806 05:00:37.405162 30028 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.96.125" port: 43181 } attempt: 4
I20260806 05:00:37.406435 30031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:00:37.405792 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.125:43181 (local address 127.0.0.1:45542)
0806 05:00:37.406153 (+ 361us) negotiation.cc:107] Waiting for socket to connect
0806 05:00:37.406276 (+ 123us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.96.125:43181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":147}
I20260806 05:00:37.407529 30031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:00:37.407078 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.125:43181 (local address 127.0.0.1:45544)
0806 05:00:37.407246 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0806 05:00:37.407341 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.96.125:43181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260806 05:00:37.407718 30028 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.96.125:43181: Network error: Client connection negotiation failed: client connection to 127.9.96.125:43181: connect: Connection refused (error 111)
W20260806 05:00:37.412904 30037 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 05:00:37.413018 30037 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 05:00:37.413118 30037 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 05:00:37.413219 30037 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 05:00:37.413283 30037 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 05:00:37.422788 30037 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.96.84:40965
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785992227050689-9601-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785992227050689-9601-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.9.96.126:36525,127.9.96.125:43181,127.9.96.124:37647
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.125.keytab
--principal=kudu/127.9.96.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.96.125:43181
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785992227050689-9601-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.9.96.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785992227050689-9601-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 03:56:12 UTC on ccycloud.quasar-guijbm.root.comops.site
build id 4569618
ASAN enabled
I20260806 05:00:37.424136 30037 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 05:00:37.425396 30037 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 05:00:37.437510 30045 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 05:00:37.437453 30050 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 05:00:37.437532 30047 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 05:00:37.443450 30037 server_base.cc:1034] running on GCE node
I20260806 05:00:37.444193 30037 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 05:00:37.445533 30037 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 05:00:37.446774 30037 hybrid_clock.cc:648] HybridClock initialized: now 1785992437446734 us; error 67 us; skew 500 ppm
I20260806 05:00:37.447124 30037 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 05:00:37 dist-test-slave-x4qh krb5kdc[29902](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992437, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.96.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 05:00:37.450254 30037 init.cc:377] Logged in from keytab as kudu/127.9.96.125@KRBTEST.COM (short username kudu)
I20260806 05:00:37.451924 30037 webserver.cc:460] Webserver started at http://127.9.96.125:39931/ using document root <none> and password file <none>
I20260806 05:00:37.452569 30037 fs_manager.cc:359] Metadata directory not provided
I20260806 05:00:37.452669 30037 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 05:00:37.452919 30037 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 05:00:37.455931 30037 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785992227050689-9601-0/minicluster-data/master-1/data/instance:
uuid: "2f74eddebd414f74b7e76745204c5dad"
format_stamp: "Formatted at 2026-08-06 05:00:37 on dist-test-slave-x4qh"
server_key: "3e607173ffc39654289b800788e074be"
server_key_iv: "c9652f8984227760736d1cdb31bf80d0"
server_key_version: "encryptionkey@0"
I20260806 05:00:37.456756 30037 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785992227050689-9601-0/minicluster-data/master-1/wal/instance:
uuid: "2f74eddebd414f74b7e76745204c5dad"
format_stamp: "Formatted at 2026-08-06 05:00:37 on dist-test-slave-x4qh"
server_key: "3e607173ffc39654289b800788e074be"
server_key_iv: "c9652f8984227760736d1cdb31bf80d0"
server_key_version: "encryptionkey@0"
I20260806 05:00:37.461959 30037 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260806 05:00:37.465270 30059 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 05:00:37.466449 30037 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 05:00:37.466631 30037 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785992227050689-9601-0/minicluster-data/master-1/data,/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785992227050689-9601-0/minicluster-data/master-1/wal
uuid: "2f74eddebd414f74b7e76745204c5dad"
format_stamp: "Formatted at 2026-08-06 05:00:37 on dist-test-slave-x4qh"
server_key: "3e607173ffc39654289b800788e074be"
server_key_iv: "c9652f8984227760736d1cdb31bf80d0"
server_key_version: "encryptionkey@0"
I20260806 05:00:37.466828 30037 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785992227050689-9601-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785992227050689-9601-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785992227050689-9601-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 05:00:37.492978 30037 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 05:00:37.496635 30037 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 05:00:37.496933 30037 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 05:00:37.520265 30037 rpc_server.cc:307] RPC server started. Bound to: 127.9.96.125:43181
I20260806 05:00:37.520298 30135 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.96.125:43181 every 8 connection(s)
I20260806 05:00:37.521577 30037 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785992227050689-9601-0/minicluster-data/master-1/data/info.pb
I20260806 05:00:37.525194 30136 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 05:00:37.529256 9601 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu as pid 30037
I20260806 05:00:37.529459 9601 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785992227050689-9601-0/minicluster-data/master-1/wal/instance
I20260806 05:00:37.530053 9601 external_mini_cluster.cc:1428] Setting key 144a5b59d5e9bc7e02b1aa2da2ca5e94
I20260806 05:00:37.532914 30136 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.96.126" port: 36525 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 05:00:37 dist-test-slave-x4qh krb5kdc[29902](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992437, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.96.125@KRBTEST.COM for kudu/127.9.96.126@KRBTEST.COM
I20260806 05:00:37.555079 30030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:00:37.536074 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43374 (local address 127.9.96.126:36525)
0806 05:00:37.536283 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0806 05:00:37.536289 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 05:00:37.537566 (+ 1277us) server_negotiation.cc:407] Connection header received
0806 05:00:37.539449 (+ 1883us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 05:00:37.539457 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 05:00:37.539548 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 05:00:37.539667 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0806 05:00:37.541006 (+ 1339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:00:37.541953 (+ 947us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:00:37.544459 (+ 2506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:00:37.544873 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:00:37.544966 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:00:37.549286 (+ 4320us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 05:00:37.549291 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 05:00:37.549295 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 05:00:37.549326 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0806 05:00:37.551980 (+ 2654us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 05:00:37.552848 (+ 868us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 05:00:37.552854 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 05:00:37.552858 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 05:00:37.552981 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 05:00:37.553409 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 05:00:37.553415 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 05:00:37.553419 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 05:00:37.553701 (+ 282us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 05:00:37.553896 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0806 05:00:37.554528 (+ 632us) server_negotiation.cc:299] Negotiation successful
0806 05:00:37.554750 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260806 05:00:37.556030 30138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:00:37.536547 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:36525 (local address 127.0.0.1:43374)
0806 05:00:37.537356 (+ 809us) negotiation.cc:107] Waiting for socket to connect
0806 05:00:37.537412 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0806 05:00:37.539169 (+ 1757us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 05:00:37.539848 (+ 679us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 05:00:37.539855 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 05:00:37.540316 (+ 461us) client_negotiation.cc:189] Negotiated authn=SASL
0806 05:00:37.540823 (+ 507us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 05:00:37.540839 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 05:00:37.542119 (+ 1280us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 05:00:37.542127 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 05:00:37.544291 (+ 2164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 05:00:37.544310 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 05:00:37.545098 (+ 788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 05:00:37.545108 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 05:00:37.545465 (+ 357us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:00:37.546187 (+ 722us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 05:00:37.546220 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0806 05:00:37.549059 (+ 2839us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 05:00:37.552125 (+ 3066us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 05:00:37.552133 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 05:00:37.552147 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0806 05:00:37.552722 (+ 575us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 05:00:37.553096 (+ 374us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 05:00:37.553101 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 05:00:37.553105 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 05:00:37.553280 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 05:00:37.553852 (+ 572us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 05:00:37.553861 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 05:00:37.554172 (+ 311us) client_negotiation.cc:769] Sending connection context
0806 05:00:37.554436 (+ 264us) client_negotiation.cc:240] Negotiation successful
0806 05:00:37.554893 (+ 457us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":565,"thread_start_us":393,"threads_started":1}
WARNING: no policy specified for kudu/127.9.96.124@KRBTEST.COM; defaulting to no policy
I20260806 05:00:37.557837 30136 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.96.125" port: 43181 } has no permanent_uuid. Determining permanent_uuid...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.96.124@KRBTEST.COM" created.
Aug 06 05:00:37 dist-test-slave-x4qh krb5kdc[29902](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992437, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.96.125@KRBTEST.COM for kudu/127.9.96.125@KRBTEST.COM
I20260806 05:00:37.572310 30138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:00:37.558480 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.125:43181 (local address 127.0.0.1:45552)
0806 05:00:37.558768 (+ 288us) negotiation.cc:107] Waiting for socket to connect
0806 05:00:37.558789 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0806 05:00:37.558877 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 05:00:37.560025 (+ 1148us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 05:00:37.560029 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 05:00:37.560468 (+ 439us) client_negotiation.cc:189] Negotiated authn=SASL
0806 05:00:37.560584 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 05:00:37.560592 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 05:00:37.561923 (+ 1331us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 05:00:37.561931 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 05:00:37.562746 (+ 815us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 05:00:37.562757 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 05:00:37.563620 (+ 863us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 05:00:37.563625 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 05:00:37.563864 (+ 239us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:00:37.564433 (+ 569us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 05:00:37.564449 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 05:00:37.566365 (+ 1916us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 05:00:37.569788 (+ 3423us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 05:00:37.569793 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 05:00:37.569797 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 05:00:37.570142 (+ 345us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 05:00:37.570520 (+ 378us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 05:00:37.570525 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 05:00:37.570528 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 05:00:37.570675 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 05:00:37.571302 (+ 627us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 05:00:37.571307 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 05:00:37.571474 (+ 167us) client_negotiation.cc:769] Sending connection context
0806 05:00:37.571615 (+ 141us) client_negotiation.cc:240] Negotiation successful
0806 05:00:37.571916 (+ 301us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":149,"mutex_wait_us":1}
I20260806 05:00:37.572417 30143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:00:37.558833 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45552 (local address 127.9.96.125:43181)
0806 05:00:37.559602 (+ 769us) server_negotiation.cc:206] Beginning negotiation
0806 05:00:37.559612 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 05:00:37.559644 (+ 32us) server_negotiation.cc:407] Connection header received
0806 05:00:37.559699 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 05:00:37.559706 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 05:00:37.559880 (+ 174us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 05:00:37.560031 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0806 05:00:37.560736 (+ 705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:00:37.561777 (+ 1041us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:00:37.562882 (+ 1105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:00:37.563472 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:00:37.563607 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:00:37.566587 (+ 2980us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 05:00:37.566598 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 05:00:37.566609 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 05:00:37.566635 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0806 05:00:37.569627 (+ 2992us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 05:00:37.570287 (+ 660us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 05:00:37.570293 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 05:00:37.570295 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 05:00:37.570405 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 05:00:37.570780 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 05:00:37.570787 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 05:00:37.570790 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 05:00:37.571169 (+ 379us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 05:00:37.571416 (+ 247us) server_negotiation.cc:1035] Waiting for connection context
0806 05:00:37.571749 (+ 333us) server_negotiation.cc:299] Negotiation successful
0806 05:00:37.572030 (+ 281us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":80,"server-negotiator.queue_time_us":588,"thread_start_us":403,"threads_started":1}
I20260806 05:00:37.574513 30136 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.96.124" port: 37647 } has no permanent_uuid. Determining permanent_uuid...
I20260806 05:00:37.575867 30138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:00:37.575210 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.124:37647 (local address 127.0.0.1:60464)
0806 05:00:37.575471 (+ 261us) negotiation.cc:107] Waiting for socket to connect
0806 05:00:37.575647 (+ 176us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.96.124:37647: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
W20260806 05:00:37.576169 30060 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.9.96.124:37647: connect: Connection refused (error 111)
I20260806 05:00:37.577864 30138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:00:37.577391 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.124:37647 (local address 127.0.0.1:60478)
0806 05:00:37.577581 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0806 05:00:37.577711 (+ 130us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.96.124:37647: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
W20260806 05:00:37.578126 30136 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.96.124:37647: Network error: Client connection negotiation failed: client connection to 127.9.96.124:37647: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.96.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.124.keytab.
Entry for principal kudu/127.9.96.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.124.keytab.
I20260806 05:00:37.590797 9601 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
/tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785992227050689-9601-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785992227050689-9601-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785992227050689-9601-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785992227050689-9601-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.96.124:37647
--webserver_interface=127.9.96.124
--webserver_port=0
--keytab_file=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.124.keytab
--principal=kudu/127.9.96.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.96.84:40965
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.96.126:36525,127.9.96.125:43181,127.9.96.124:37647
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785992227050689-9601-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785992227050689-9601-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785992227050689-9601-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 05:00:37.627485 30136 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.96.124" port: 37647 } attempt: 1
I20260806 05:00:37.628737 30138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:00:37.628212 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.124:37647 (local address 127.0.0.1:60480)
0806 05:00:37.628432 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0806 05:00:37.628556 (+ 124us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.96.124:37647: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
I20260806 05:00:37.629760 30138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:00:37.629364 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.124:37647 (local address 127.0.0.1:60484)
0806 05:00:37.629543 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0806 05:00:37.629632 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.96.124:37647: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
W20260806 05:00:37.630002 30136 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.96.124:37647: Network error: Client connection negotiation failed: client connection to 127.9.96.124:37647: connect: Connection refused (error 111)
I20260806 05:00:37.698246 30136 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.96.124" port: 37647 } attempt: 2
I20260806 05:00:37.699692 30138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:00:37.698985 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.124:37647 (local address 127.0.0.1:60488)
0806 05:00:37.699309 (+ 324us) negotiation.cc:107] Waiting for socket to connect
0806 05:00:37.699467 (+ 158us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.96.124:37647: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":95}
I20260806 05:00:37.700785 30138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:00:37.700391 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.124:37647 (local address 127.0.0.1:60498)
0806 05:00:37.700560 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0806 05:00:37.700644 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.96.124:37647: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
W20260806 05:00:37.701058 30136 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.96.124:37647: Network error: Client connection negotiation failed: client connection to 127.9.96.124:37647: connect: Connection refused (error 111)
I20260806 05:00:37.707073 30028 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.96.125" port: 43181 } attempt: 5
Aug 06 05:00:37 dist-test-slave-x4qh krb5kdc[29902](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992436, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.96.126@KRBTEST.COM for kudu/127.9.96.125@KRBTEST.COM
I20260806 05:00:37.720760 30143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:00:37.707907 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45566 (local address 127.9.96.125:43181)
0806 05:00:37.708070 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0806 05:00:37.708075 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 05:00:37.708098 (+ 23us) server_negotiation.cc:407] Connection header received
0806 05:00:37.708173 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 05:00:37.708178 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 05:00:37.708263 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 05:00:37.708380 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0806 05:00:37.709179 (+ 799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:00:37.709971 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:00:37.711140 (+ 1169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:00:37.711697 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:00:37.711853 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:00:37.715550 (+ 3697us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 05:00:37.715554 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 05:00:37.715558 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 05:00:37.715588 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0806 05:00:37.718116 (+ 2528us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 05:00:37.718904 (+ 788us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 05:00:37.718908 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 05:00:37.718910 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 05:00:37.719024 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 05:00:37.719459 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 05:00:37.719462 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 05:00:37.719464 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 05:00:37.719807 (+ 343us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 05:00:37.719917 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0806 05:00:37.720383 (+ 466us) server_negotiation.cc:299] Negotiation successful
0806 05:00:37.720565 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260806 05:00:37.720899 30031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:00:37.707708 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.125:43181 (local address 127.0.0.1:45566)
0806 05:00:37.707898 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0806 05:00:37.707922 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0806 05:00:37.708062 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 05:00:37.708429 (+ 367us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 05:00:37.708434 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 05:00:37.708850 (+ 416us) client_negotiation.cc:189] Negotiated authn=SASL
0806 05:00:37.709033 (+ 183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 05:00:37.709042 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 05:00:37.710122 (+ 1080us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 05:00:37.710129 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 05:00:37.710997 (+ 868us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 05:00:37.711009 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 05:00:37.711924 (+ 915us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 05:00:37.711930 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 05:00:37.712100 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:00:37.712761 (+ 661us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 05:00:37.712778 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 05:00:37.715329 (+ 2551us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 05:00:37.718257 (+ 2928us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 05:00:37.718263 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 05:00:37.718267 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 05:00:37.718785 (+ 518us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 05:00:37.719136 (+ 351us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 05:00:37.719141 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 05:00:37.719144 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 05:00:37.719332 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 05:00:37.719935 (+ 603us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 05:00:37.719940 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 05:00:37.720104 (+ 164us) client_negotiation.cc:769] Sending connection context
0806 05:00:37.720288 (+ 184us) client_negotiation.cc:240] Negotiation successful
0806 05:00:37.720559 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":61}
I20260806 05:00:37.722088 30028 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.96.124" port: 37647 } has no permanent_uuid. Determining permanent_uuid...
I20260806 05:00:37.723202 30031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:00:37.722763 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.124:37647 (local address 127.0.0.1:60502)
0806 05:00:37.722952 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0806 05:00:37.723042 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.96.124:37647: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
I20260806 05:00:37.724346 30031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:00:37.723885 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.124:37647 (local address 127.0.0.1:60506)
0806 05:00:37.724088 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0806 05:00:37.724198 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.96.124:37647: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260806 05:00:37.724654 30028 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.96.124:37647: Network error: Client connection negotiation failed: client connection to 127.9.96.124:37647: connect: Connection refused (error 111)
I20260806 05:00:37.775944 30028 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.96.124" port: 37647 } attempt: 1
I20260806 05:00:37.777295 30031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:00:37.776681 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.124:37647 (local address 127.0.0.1:60510)
0806 05:00:37.776915 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0806 05:00:37.777052 (+ 137us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.96.124:37647: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":79}
I20260806 05:00:37.778390 30031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:00:37.778018 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.124:37647 (local address 127.0.0.1:60522)
0806 05:00:37.778173 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0806 05:00:37.778260 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.96.124:37647: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260806 05:00:37.778605 30028 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.96.124:37647: Network error: Client connection negotiation failed: client connection to 127.9.96.124:37647: connect: Connection refused (error 111)
I20260806 05:00:37.792296 30136 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.96.124" port: 37647 } attempt: 3
I20260806 05:00:37.793390 30138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:00:37.792905 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.124:37647 (local address 127.0.0.1:60534)
0806 05:00:37.793107 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0806 05:00:37.793209 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.96.124:37647: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
I20260806 05:00:37.794507 30138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:00:37.794032 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.124:37647 (local address 127.0.0.1:60544)
0806 05:00:37.794271 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0806 05:00:37.794363 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.96.124:37647: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":135}
W20260806 05:00:37.794747 30136 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.96.124:37647: Network error: Client connection negotiation failed: client connection to 127.9.96.124:37647: connect: Connection refused (error 111)
I20260806 05:00:37.846846 30028 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.96.124" port: 37647 } attempt: 2
I20260806 05:00:37.848261 30031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:00:37.847601 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.124:37647 (local address 127.0.0.1:60548)
0806 05:00:37.847872 (+ 271us) negotiation.cc:107] Waiting for socket to connect
0806 05:00:37.848007 (+ 135us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.96.124:37647: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74}
I20260806 05:00:37.849392 30031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:00:37.848965 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.124:37647 (local address 127.0.0.1:60560)
0806 05:00:37.849138 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0806 05:00:37.849253 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.96.124:37647: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260806 05:00:37.849628 30028 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.96.124:37647: Network error: Client connection negotiation failed: client connection to 127.9.96.124:37647: connect: Connection refused (error 111)
W20260806 05:00:37.933451 30146 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 05:00:37.933758 30146 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 05:00:37.933857 30146 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260806 05:00:37.938050 30136 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.96.124" port: 37647 } attempt: 4
I20260806 05:00:37.939296 30138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:00:37.938789 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.124:37647 (local address 127.0.0.1:60568)
0806 05:00:37.939045 (+ 256us) negotiation.cc:107] Waiting for socket to connect
0806 05:00:37.939136 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.96.124:37647: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":80}
I20260806 05:00:37.940330 30138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:00:37.940017 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.124:37647 (local address 127.0.0.1:60576)
0806 05:00:37.940148 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0806 05:00:37.940216 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.96.124:37647: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260806 05:00:37.940616 30136 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.96.124:37647: Network error: Client connection negotiation failed: client connection to 127.9.96.124:37647: connect: Connection refused (error 111)
W20260806 05:00:37.942651 30146 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 05:00:37.942800 30146 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 05:00:37.942901 30146 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 05:00:37.943009 30146 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 05:00:37.943095 30146 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 05:00:37.953696 30146 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.96.84:40965
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785992227050689-9601-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785992227050689-9601-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.9.96.126:36525,127.9.96.125:43181,127.9.96.124:37647
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.124.keytab
--principal=kudu/127.9.96.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.96.124:37647
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785992227050689-9601-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.9.96.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785992227050689-9601-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 03:56:12 UTC on ccycloud.quasar-guijbm.root.comops.site
build id 4569618
ASAN enabled
I20260806 05:00:37.955006 30146 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 05:00:37.955854 30028 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.96.124" port: 37647 } attempt: 3
I20260806 05:00:37.956560 30146 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 05:00:37.957013 30031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:00:37.956479 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.124:37647 (local address 127.0.0.1:60590)
0806 05:00:37.956721 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0806 05:00:37.956826 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.96.124:37647: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
I20260806 05:00:37.958153 30031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:00:37.957732 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.124:37647 (local address 127.0.0.1:60600)
0806 05:00:37.957939 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0806 05:00:37.958027 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.96.124:37647: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":90}
W20260806 05:00:37.958405 30028 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.96.124:37647: Network error: Client connection negotiation failed: client connection to 127.9.96.124:37647: connect: Connection refused (error 111)
W20260806 05:00:37.969247 30155 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 05:00:37.969328 30154 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 05:00:37.969304 30157 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 05:00:37.969906 30146 server_base.cc:1034] running on GCE node
I20260806 05:00:37.970793 30146 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 05:00:37.972198 30146 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 05:00:37.973394 30146 hybrid_clock.cc:648] HybridClock initialized: now 1785992437973330 us; error 61 us; skew 500 ppm
I20260806 05:00:37.973740 30146 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 05:00:37 dist-test-slave-x4qh krb5kdc[29902](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992437, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.96.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 05:00:37.976843 30146 init.cc:377] Logged in from keytab as kudu/127.9.96.124@KRBTEST.COM (short username kudu)
I20260806 05:00:37.978282 30146 webserver.cc:460] Webserver started at http://127.9.96.124:46587/ using document root <none> and password file <none>
I20260806 05:00:37.978971 30146 fs_manager.cc:359] Metadata directory not provided
I20260806 05:00:37.979099 30146 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 05:00:37.979470 30146 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 05:00:37.982391 30146 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785992227050689-9601-0/minicluster-data/master-2/data/instance:
uuid: "76e710aebc494dfaa24f0ba592902b75"
format_stamp: "Formatted at 2026-08-06 05:00:37 on dist-test-slave-x4qh"
server_key: "fed46f7a743dcac770b3c04d0f49ff46"
server_key_iv: "055e88dcfecea66e134a29a4f13fac5b"
server_key_version: "encryptionkey@0"
I20260806 05:00:37.983208 30146 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785992227050689-9601-0/minicluster-data/master-2/wal/instance:
uuid: "76e710aebc494dfaa24f0ba592902b75"
format_stamp: "Formatted at 2026-08-06 05:00:37 on dist-test-slave-x4qh"
server_key: "fed46f7a743dcac770b3c04d0f49ff46"
server_key_iv: "055e88dcfecea66e134a29a4f13fac5b"
server_key_version: "encryptionkey@0"
I20260806 05:00:37.988200 30146 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.002s
I20260806 05:00:37.991506 30169 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 05:00:37.992692 30146 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260806 05:00:37.992874 30146 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785992227050689-9601-0/minicluster-data/master-2/data,/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785992227050689-9601-0/minicluster-data/master-2/wal
uuid: "76e710aebc494dfaa24f0ba592902b75"
format_stamp: "Formatted at 2026-08-06 05:00:37 on dist-test-slave-x4qh"
server_key: "fed46f7a743dcac770b3c04d0f49ff46"
server_key_iv: "055e88dcfecea66e134a29a4f13fac5b"
server_key_version: "encryptionkey@0"
I20260806 05:00:37.993065 30146 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785992227050689-9601-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785992227050689-9601-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785992227050689-9601-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 05:00:38.017025 30146 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 05:00:38.020991 30146 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 05:00:38.021284 30146 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 05:00:38.040112 30146 rpc_server.cc:307] RPC server started. Bound to: 127.9.96.124:37647
I20260806 05:00:38.040146 30245 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.96.124:37647 every 8 connection(s)
I20260806 05:00:38.041406 30146 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785992227050689-9601-0/minicluster-data/master-2/data/info.pb
I20260806 05:00:38.045174 30246 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 05:00:38.045214 9601 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu as pid 30146
I20260806 05:00:38.045419 9601 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785992227050689-9601-0/minicluster-data/master-2/wal/instance
I20260806 05:00:38.045930 9601 external_mini_cluster.cc:1428] Setting key d4fe45505e17e0ed5a99ea672563d56c
I20260806 05:00:38.054096 30246 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.96.126" port: 36525 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 05:00:38 dist-test-slave-x4qh krb5kdc[29902](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992436, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.96.124@KRBTEST.COM
Aug 06 05:00:38 dist-test-slave-x4qh krb5kdc[29902](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992437, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.96.124@KRBTEST.COM for kudu/127.9.96.126@KRBTEST.COM
I20260806 05:00:38.071497 30250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:00:38.047847 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60610 (local address 127.9.96.124:37647)
0806 05:00:38.048667 (+ 820us) server_negotiation.cc:206] Beginning negotiation
0806 05:00:38.048678 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 05:00:38.048718 (+ 40us) server_negotiation.cc:407] Connection header received
0806 05:00:38.050946 (+ 2228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 05:00:38.050968 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 05:00:38.051320 (+ 352us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 05:00:38.051699 (+ 379us) server_negotiation.cc:226] Negotiated authn=SASL
0806 05:00:38.053624 (+ 1925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:00:38.054954 (+ 1330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:00:38.056765 (+ 1811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:00:38.057836 (+ 1071us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:00:38.058879 (+ 1043us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:00:38.063227 (+ 4348us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 05:00:38.063234 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 05:00:38.063243 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 05:00:38.063281 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0806 05:00:38.066989 (+ 3708us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 05:00:38.067819 (+ 830us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 05:00:38.067828 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 05:00:38.067836 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0806 05:00:38.068022 (+ 186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 05:00:38.068404 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 05:00:38.068408 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 05:00:38.068410 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 05:00:38.068915 (+ 505us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 05:00:38.069228 (+ 313us) server_negotiation.cc:1035] Waiting for connection context
0806 05:00:38.069693 (+ 465us) server_negotiation.cc:299] Negotiation successful
0806 05:00:38.070154 (+ 461us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":628,"thread_start_us":415,"threads_started":1}
I20260806 05:00:38.073639 30252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:00:38.056856 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43388 (local address 127.9.96.126:36525)
0806 05:00:38.057514 (+ 658us) server_negotiation.cc:206] Beginning negotiation
0806 05:00:38.057520 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 05:00:38.058294 (+ 774us) server_negotiation.cc:407] Connection header received
0806 05:00:38.058409 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 05:00:38.058415 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 05:00:38.058490 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 05:00:38.058612 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0806 05:00:38.060322 (+ 1710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:00:38.061470 (+ 1148us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:00:38.062895 (+ 1425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:00:38.063487 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:00:38.063598 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:00:38.067515 (+ 3917us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 05:00:38.067520 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 05:00:38.067523 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 05:00:38.067555 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0806 05:00:38.070572 (+ 3017us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 05:00:38.071461 (+ 889us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 05:00:38.071465 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 05:00:38.071467 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 05:00:38.071651 (+ 184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 05:00:38.072084 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 05:00:38.072089 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 05:00:38.072092 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 05:00:38.072377 (+ 285us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 05:00:38.072536 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0806 05:00:38.072999 (+ 463us) server_negotiation.cc:299] Negotiation successful
0806 05:00:38.073209 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":499,"thread_start_us":308,"threads_started":1}
I20260806 05:00:38.073661 30253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:00:38.056862 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:36525 (local address 127.0.0.1:43388)
0806 05:00:38.057986 (+ 1124us) negotiation.cc:107] Waiting for socket to connect
0806 05:00:38.058030 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0806 05:00:38.058189 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 05:00:38.058837 (+ 648us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 05:00:38.058843 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 05:00:38.059733 (+ 890us) client_negotiation.cc:189] Negotiated authn=SASL
0806 05:00:38.060112 (+ 379us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 05:00:38.060125 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 05:00:38.061642 (+ 1517us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 05:00:38.061647 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 05:00:38.062733 (+ 1086us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 05:00:38.062746 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 05:00:38.063673 (+ 927us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 05:00:38.063679 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 05:00:38.063882 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:00:38.064477 (+ 595us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 05:00:38.064505 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0806 05:00:38.067262 (+ 2757us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 05:00:38.070712 (+ 3450us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 05:00:38.070721 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 05:00:38.070728 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0806 05:00:38.071297 (+ 569us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 05:00:38.071789 (+ 492us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 05:00:38.071794 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 05:00:38.071798 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 05:00:38.071957 (+ 159us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 05:00:38.072511 (+ 554us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 05:00:38.072518 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 05:00:38.072694 (+ 176us) client_negotiation.cc:769] Sending connection context
0806 05:00:38.072879 (+ 185us) client_negotiation.cc:240] Negotiation successful
0806 05:00:38.073194 (+ 315us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":847,"thread_start_us":679,"threads_started":1}
I20260806 05:00:38.075462 30246 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.96.125" port: 43181 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 05:00:38 dist-test-slave-x4qh krb5kdc[29902](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992437, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.96.124@KRBTEST.COM for kudu/127.9.96.125@KRBTEST.COM
Aug 06 05:00:38 dist-test-slave-x4qh krb5kdc[29902](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992436, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.96.126@KRBTEST.COM
I20260806 05:00:38.089396 30253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:00:38.076125 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.125:43181 (local address 127.0.0.1:45570)
0806 05:00:38.076257 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0806 05:00:38.076274 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 05:00:38.076408 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 05:00:38.076760 (+ 352us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 05:00:38.076764 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 05:00:38.077128 (+ 364us) client_negotiation.cc:189] Negotiated authn=SASL
0806 05:00:38.077263 (+ 135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 05:00:38.077270 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 05:00:38.078253 (+ 983us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 05:00:38.078258 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 05:00:38.079103 (+ 845us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 05:00:38.079114 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 05:00:38.079968 (+ 854us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 05:00:38.079976 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 05:00:38.080239 (+ 263us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:00:38.080719 (+ 480us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 05:00:38.080743 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0806 05:00:38.083158 (+ 2415us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 05:00:38.086904 (+ 3746us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 05:00:38.086908 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 05:00:38.086910 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 05:00:38.087260 (+ 350us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 05:00:38.087703 (+ 443us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 05:00:38.087709 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 05:00:38.087712 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 05:00:38.087913 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 05:00:38.088528 (+ 615us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 05:00:38.088533 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 05:00:38.088707 (+ 174us) client_negotiation.cc:769] Sending connection context
0806 05:00:38.088819 (+ 112us) client_negotiation.cc:240] Negotiation successful
0806 05:00:38.089011 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34}
I20260806 05:00:38.089555 30143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:00:38.076271 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45570 (local address 127.9.96.125:43181)
0806 05:00:38.076492 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0806 05:00:38.076499 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 05:00:38.076516 (+ 17us) server_negotiation.cc:407] Connection header received
0806 05:00:38.076574 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 05:00:38.076578 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 05:00:38.076634 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 05:00:38.076737 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0806 05:00:38.077427 (+ 690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:00:38.078102 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:00:38.079241 (+ 1139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:00:38.079778 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:00:38.079925 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:00:38.083342 (+ 3417us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 05:00:38.083347 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 05:00:38.083350 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 05:00:38.083449 (+ 99us) server_negotiation.cc:922] calling sasl_server_start()
0806 05:00:38.086774 (+ 3325us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 05:00:38.087423 (+ 649us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 05:00:38.087428 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 05:00:38.087431 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 05:00:38.087588 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 05:00:38.088027 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 05:00:38.088030 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 05:00:38.088032 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 05:00:38.088394 (+ 362us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 05:00:38.088517 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0806 05:00:38.088943 (+ 426us) server_negotiation.cc:299] Negotiation successful
0806 05:00:38.089169 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260806 05:00:38.090399 30252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:00:38.074568 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43394 (local address 127.9.96.126:36525)
0806 05:00:38.074762 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0806 05:00:38.074768 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 05:00:38.074861 (+ 93us) server_negotiation.cc:407] Connection header received
0806 05:00:38.075001 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 05:00:38.075006 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 05:00:38.075099 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 05:00:38.075206 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0806 05:00:38.076778 (+ 1572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:00:38.077480 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:00:38.078757 (+ 1277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:00:38.079167 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:00:38.079303 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:00:38.084052 (+ 4749us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 05:00:38.084056 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 05:00:38.084059 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 05:00:38.084088 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0806 05:00:38.087290 (+ 3202us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 05:00:38.088137 (+ 847us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 05:00:38.088142 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 05:00:38.088145 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 05:00:38.088276 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 05:00:38.088691 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 05:00:38.088694 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 05:00:38.088696 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 05:00:38.089048 (+ 352us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 05:00:38.089306 (+ 258us) server_negotiation.cc:1035] Waiting for connection context
0806 05:00:38.089852 (+ 546us) server_negotiation.cc:299] Negotiation successful
0806 05:00:38.090075 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260806 05:00:38.090415 30246 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.96.124" port: 37647 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 05:00:38 dist-test-slave-x4qh krb5kdc[29902](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992437, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.96.124@KRBTEST.COM for kudu/127.9.96.124@KRBTEST.COM
Aug 06 05:00:38 dist-test-slave-x4qh krb5kdc[29902](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992436, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.96.125@KRBTEST.COM
I20260806 05:00:38.104521 30250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:00:38.091167 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60624 (local address 127.9.96.124:37647)
0806 05:00:38.091429 (+ 262us) server_negotiation.cc:206] Beginning negotiation
0806 05:00:38.091437 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 05:00:38.091632 (+ 195us) server_negotiation.cc:407] Connection header received
0806 05:00:38.091761 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 05:00:38.091764 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 05:00:38.091816 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 05:00:38.091916 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0806 05:00:38.092752 (+ 836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:00:38.093477 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:00:38.094784 (+ 1307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:00:38.095233 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:00:38.095321 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:00:38.098764 (+ 3443us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 05:00:38.098771 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 05:00:38.098776 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 05:00:38.098825 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0806 05:00:38.101830 (+ 3005us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 05:00:38.102594 (+ 764us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 05:00:38.102602 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 05:00:38.102605 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 05:00:38.102720 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 05:00:38.103073 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 05:00:38.103080 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 05:00:38.103083 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 05:00:38.103348 (+ 265us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 05:00:38.103579 (+ 231us) server_negotiation.cc:1035] Waiting for connection context
0806 05:00:38.103919 (+ 340us) server_negotiation.cc:299] Negotiation successful
0806 05:00:38.104134 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260806 05:00:38.104727 30253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:00:38.091195 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.124:37647 (local address 127.0.0.1:60624)
0806 05:00:38.091520 (+ 325us) negotiation.cc:107] Waiting for socket to connect
0806 05:00:38.091539 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0806 05:00:38.091663 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 05:00:38.091946 (+ 283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 05:00:38.091950 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 05:00:38.092426 (+ 476us) client_negotiation.cc:189] Negotiated authn=SASL
0806 05:00:38.092617 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 05:00:38.092630 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 05:00:38.093624 (+ 994us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 05:00:38.093631 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 05:00:38.094646 (+ 1015us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 05:00:38.094658 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 05:00:38.095445 (+ 787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 05:00:38.095452 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 05:00:38.095651 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:00:38.096215 (+ 564us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 05:00:38.096237 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0806 05:00:38.098584 (+ 2347us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 05:00:38.101952 (+ 3368us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 05:00:38.101959 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 05:00:38.101962 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 05:00:38.102466 (+ 504us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 05:00:38.102827 (+ 361us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 05:00:38.102833 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 05:00:38.102836 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 05:00:38.102968 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 05:00:38.103544 (+ 576us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 05:00:38.103549 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 05:00:38.103687 (+ 138us) client_negotiation.cc:769] Sending connection context
0806 05:00:38.103844 (+ 157us) client_negotiation.cc:240] Negotiation successful
0806 05:00:38.104290 (+ 446us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":224,"mutex_wait_us":145}
I20260806 05:00:38.107690 30143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:00:38.092265 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45580 (local address 127.9.96.125:43181)
0806 05:00:38.092435 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0806 05:00:38.092439 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 05:00:38.092454 (+ 15us) server_negotiation.cc:407] Connection header received
0806 05:00:38.092625 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 05:00:38.092628 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 05:00:38.092696 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 05:00:38.092795 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0806 05:00:38.094036 (+ 1241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:00:38.094697 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:00:38.096456 (+ 1759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:00:38.096877 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:00:38.096968 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:00:38.101829 (+ 4861us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 05:00:38.101833 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 05:00:38.101836 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 05:00:38.101866 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0806 05:00:38.104797 (+ 2931us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 05:00:38.105643 (+ 846us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 05:00:38.105648 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 05:00:38.105651 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 05:00:38.105814 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 05:00:38.106216 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 05:00:38.106222 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 05:00:38.106225 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 05:00:38.106488 (+ 263us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 05:00:38.106770 (+ 282us) server_negotiation.cc:1035] Waiting for connection context
0806 05:00:38.107097 (+ 327us) server_negotiation.cc:299] Negotiation successful
0806 05:00:38.107327 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260806 05:00:38.109880 30246 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 76e710aebc494dfaa24f0ba592902b75: Bootstrap starting.
I20260806 05:00:38.112527 9601 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 05:00:38.113487 30246 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 76e710aebc494dfaa24f0ba592902b75: Neither blocks nor log segments found. Creating new log.
I20260806 05:00:38.114518 30246 log.cc:824] T 00000000000000000000000000000000 P 76e710aebc494dfaa24f0ba592902b75: Log is configured to *not* fsync() on all Append() calls
I20260806 05:00:38.117475 30246 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 76e710aebc494dfaa24f0ba592902b75: No bootstrap required, opened a new log
I20260806 05:00:38.129192 30246 raft_consensus.cc:348] T 00000000000000000000000000000000 P 76e710aebc494dfaa24f0ba592902b75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd7bd28b7a2e47c48fdbf033821161af" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 36525 } } peers { permanent_uuid: "2f74eddebd414f74b7e76745204c5dad" member_type: VOTER last_known_addr { host: "127.9.96.125" port: 43181 } } peers { permanent_uuid: "76e710aebc494dfaa24f0ba592902b75" member_type: VOTER last_known_addr { host: "127.9.96.124" port: 37647 } }
I20260806 05:00:38.129777 30246 raft_consensus.cc:374] T 00000000000000000000000000000000 P 76e710aebc494dfaa24f0ba592902b75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 05:00:38.129987 30246 raft_consensus.cc:729] T 00000000000000000000000000000000 P 76e710aebc494dfaa24f0ba592902b75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 76e710aebc494dfaa24f0ba592902b75, State: Initialized, Role: FOLLOWER
I20260806 05:00:38.130889 30246 consensus_queue.cc:260] T 00000000000000000000000000000000 P 76e710aebc494dfaa24f0ba592902b75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd7bd28b7a2e47c48fdbf033821161af" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 36525 } } peers { permanent_uuid: "2f74eddebd414f74b7e76745204c5dad" member_type: VOTER last_known_addr { host: "127.9.96.125" port: 43181 } } peers { permanent_uuid: "76e710aebc494dfaa24f0ba592902b75" member_type: VOTER last_known_addr { host: "127.9.96.124" port: 37647 } }
I20260806 05:00:38.135659 30028 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.96.124" port: 37647 } attempt: 4
I20260806 05:00:38.138062 30265 sys_catalog.cc:455] T 00000000000000000000000000000000 P 76e710aebc494dfaa24f0ba592902b75 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd7bd28b7a2e47c48fdbf033821161af" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 36525 } } peers { permanent_uuid: "2f74eddebd414f74b7e76745204c5dad" member_type: VOTER last_known_addr { host: "127.9.96.125" port: 43181 } } peers { permanent_uuid: "76e710aebc494dfaa24f0ba592902b75" member_type: VOTER last_known_addr { host: "127.9.96.124" port: 37647 } } }
I20260806 05:00:38.139585 30252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:00:38.117483 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43400 (local address 127.9.96.126:36525)
0806 05:00:38.117677 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0806 05:00:38.117683 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 05:00:38.118467 (+ 784us) server_negotiation.cc:407] Connection header received
0806 05:00:38.118783 (+ 316us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 05:00:38.118791 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 05:00:38.118876 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 05:00:38.118955 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0806 05:00:38.121389 (+ 2434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:00:38.122498 (+ 1109us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:00:38.124706 (+ 2208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:00:38.125393 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:00:38.125495 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:00:38.129642 (+ 4147us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 05:00:38.129648 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 05:00:38.129652 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 05:00:38.129690 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0806 05:00:38.133874 (+ 4184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 05:00:38.135810 (+ 1936us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 05:00:38.135817 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 05:00:38.135820 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 05:00:38.135998 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 05:00:38.136836 (+ 838us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 05:00:38.136843 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 05:00:38.136849 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 05:00:38.137303 (+ 454us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 05:00:38.137625 (+ 322us) server_negotiation.cc:1035] Waiting for connection context
0806 05:00:38.138921 (+ 1296us) server_negotiation.cc:299] Negotiation successful
0806 05:00:38.139141 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260806 05:00:38.140846 30265 sys_catalog.cc:458] T 00000000000000000000000000000000 P 76e710aebc494dfaa24f0ba592902b75 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 05:00:38.140707 30246 sys_catalog.cc:564] T 00000000000000000000000000000000 P 76e710aebc494dfaa24f0ba592902b75 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 05:00:38.149649 30250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:00:38.117556 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60640 (local address 127.9.96.124:37647)
0806 05:00:38.117863 (+ 307us) server_negotiation.cc:206] Beginning negotiation
0806 05:00:38.117869 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 05:00:38.126540 (+ 8671us) server_negotiation.cc:407] Connection header received
0806 05:00:38.127066 (+ 526us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 05:00:38.127075 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 05:00:38.127178 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 05:00:38.127267 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0806 05:00:38.130039 (+ 2772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:00:38.131559 (+ 1520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:00:38.134256 (+ 2697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:00:38.135234 (+ 978us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:00:38.135345 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:00:38.139343 (+ 3998us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 05:00:38.139352 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 05:00:38.139360 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 05:00:38.139448 (+ 88us) server_negotiation.cc:922] calling sasl_server_start()
0806 05:00:38.145849 (+ 6401us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 05:00:38.146798 (+ 949us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 05:00:38.146804 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 05:00:38.146808 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 05:00:38.147009 (+ 201us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 05:00:38.147600 (+ 591us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 05:00:38.147605 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 05:00:38.147609 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 05:00:38.147999 (+ 390us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 05:00:38.148281 (+ 282us) server_negotiation.cc:1035] Waiting for connection context
0806 05:00:38.149003 (+ 722us) server_negotiation.cc:299] Negotiation successful
0806 05:00:38.149234 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":148}
Aug 06 05:00:38 dist-test-slave-x4qh krb5kdc[29902](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992436, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.96.126@KRBTEST.COM for kudu/127.9.96.124@KRBTEST.COM
I20260806 05:00:38.157586 30143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:00:38.118612 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45596 (local address 127.9.96.125:43181)
0806 05:00:38.118816 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0806 05:00:38.118824 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 05:00:38.135690 (+ 16866us) server_negotiation.cc:407] Connection header received
0806 05:00:38.138210 (+ 2520us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 05:00:38.138216 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 05:00:38.138307 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 05:00:38.138393 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0806 05:00:38.141709 (+ 3316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:00:38.142970 (+ 1261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:00:38.145327 (+ 2357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:00:38.146002 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:00:38.146135 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:00:38.149502 (+ 3367us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 05:00:38.149510 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 05:00:38.149515 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 05:00:38.149559 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0806 05:00:38.153785 (+ 4226us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 05:00:38.154785 (+ 1000us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 05:00:38.154791 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 05:00:38.154794 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 05:00:38.154969 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 05:00:38.155409 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 05:00:38.155417 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 05:00:38.155423 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 05:00:38.155849 (+ 426us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 05:00:38.156928 (+ 1079us) server_negotiation.cc:1035] Waiting for connection context
0806 05:00:38.157080 (+ 152us) server_negotiation.cc:299] Negotiation successful
0806 05:00:38.157269 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260806 05:00:38.165171 30031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:00:38.136196 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.124:37647 (local address 127.0.0.1:60648)
0806 05:00:38.136594 (+ 398us) negotiation.cc:107] Waiting for socket to connect
0806 05:00:38.136613 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0806 05:00:38.136716 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 05:00:38.145488 (+ 8772us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 05:00:38.145495 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 05:00:38.146065 (+ 570us) client_negotiation.cc:189] Negotiated authn=SASL
0806 05:00:38.146324 (+ 259us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 05:00:38.146340 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 05:00:38.148480 (+ 2140us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 05:00:38.148489 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 05:00:38.150395 (+ 1906us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 05:00:38.150417 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 05:00:38.151797 (+ 1380us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 05:00:38.151803 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 05:00:38.151996 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:00:38.152620 (+ 624us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 05:00:38.152643 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0806 05:00:38.155860 (+ 3217us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 05:00:38.160708 (+ 4848us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 05:00:38.160711 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 05:00:38.160714 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 05:00:38.161162 (+ 448us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 05:00:38.161686 (+ 524us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 05:00:38.161691 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 05:00:38.161694 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 05:00:38.161890 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 05:00:38.164138 (+ 2248us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 05:00:38.164142 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 05:00:38.164322 (+ 180us) client_negotiation.cc:769] Sending connection context
0806 05:00:38.164483 (+ 161us) client_negotiation.cc:240] Negotiation successful
0806 05:00:38.164767 (+ 284us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":270}
I20260806 05:00:38.165228 30266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:00:38.136940 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60648 (local address 127.9.96.124:37647)
0806 05:00:38.145165 (+ 8225us) server_negotiation.cc:206] Beginning negotiation
0806 05:00:38.145171 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 05:00:38.145196 (+ 25us) server_negotiation.cc:407] Connection header received
0806 05:00:38.145250 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 05:00:38.145257 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 05:00:38.145324 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 05:00:38.145461 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0806 05:00:38.146602 (+ 1141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:00:38.148243 (+ 1641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:00:38.150646 (+ 2403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:00:38.151617 (+ 971us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:00:38.151776 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:00:38.156104 (+ 4328us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 05:00:38.156110 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 05:00:38.156116 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 05:00:38.156156 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0806 05:00:38.160564 (+ 4408us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 05:00:38.161287 (+ 723us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 05:00:38.161293 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 05:00:38.161298 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 05:00:38.161495 (+ 197us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 05:00:38.163495 (+ 2000us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 05:00:38.163501 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 05:00:38.163504 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 05:00:38.164013 (+ 509us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 05:00:38.164162 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0806 05:00:38.164582 (+ 420us) server_negotiation.cc:299] Negotiation successful
0806 05:00:38.164781 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8060,"thread_start_us":7849,"threads_started":1}
I20260806 05:00:38.170032 30028 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dd7bd28b7a2e47c48fdbf033821161af: Bootstrap starting.
I20260806 05:00:38.173985 30028 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P dd7bd28b7a2e47c48fdbf033821161af: Neither blocks nor log segments found. Creating new log.
I20260806 05:00:38.175004 30028 log.cc:824] T 00000000000000000000000000000000 P dd7bd28b7a2e47c48fdbf033821161af: Log is configured to *not* fsync() on all Append() calls
I20260806 05:00:38.163138 30146 master_runner.cc:186] Error getting master registration for 127.9.96.126:36525: OK, instance_id { permanent_uuid: "dd7bd28b7a2e47c48fdbf033821161af" instance_seqno: 1785992436998946 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260806 05:00:38.177162 30146 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260806 05:00:38.178288 30283 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 76e710aebc494dfaa24f0ba592902b75: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 05:00:38.178541 30283 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 05:00:38.187271 30028 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dd7bd28b7a2e47c48fdbf033821161af: No bootstrap required, opened a new log
I20260806 05:00:38.194887 30028 raft_consensus.cc:348] T 00000000000000000000000000000000 P dd7bd28b7a2e47c48fdbf033821161af [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd7bd28b7a2e47c48fdbf033821161af" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 36525 } } peers { permanent_uuid: "2f74eddebd414f74b7e76745204c5dad" member_type: VOTER last_known_addr { host: "127.9.96.125" port: 43181 } } peers { permanent_uuid: "76e710aebc494dfaa24f0ba592902b75" member_type: VOTER last_known_addr { host: "127.9.96.124" port: 37647 } }
I20260806 05:00:38.195209 30028 raft_consensus.cc:374] T 00000000000000000000000000000000 P dd7bd28b7a2e47c48fdbf033821161af [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 05:00:38.195343 30028 raft_consensus.cc:729] T 00000000000000000000000000000000 P dd7bd28b7a2e47c48fdbf033821161af [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dd7bd28b7a2e47c48fdbf033821161af, State: Initialized, Role: FOLLOWER
I20260806 05:00:38.195977 30028 consensus_queue.cc:260] T 00000000000000000000000000000000 P dd7bd28b7a2e47c48fdbf033821161af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd7bd28b7a2e47c48fdbf033821161af" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 36525 } } peers { permanent_uuid: "2f74eddebd414f74b7e76745204c5dad" member_type: VOTER last_known_addr { host: "127.9.96.125" port: 43181 } } peers { permanent_uuid: "76e710aebc494dfaa24f0ba592902b75" member_type: VOTER last_known_addr { host: "127.9.96.124" port: 37647 } }
I20260806 05:00:38.196947 30287 sys_catalog.cc:455] T 00000000000000000000000000000000 P dd7bd28b7a2e47c48fdbf033821161af [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd7bd28b7a2e47c48fdbf033821161af" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 36525 } } peers { permanent_uuid: "2f74eddebd414f74b7e76745204c5dad" member_type: VOTER last_known_addr { host: "127.9.96.125" port: 43181 } } peers { permanent_uuid: "76e710aebc494dfaa24f0ba592902b75" member_type: VOTER last_known_addr { host: "127.9.96.124" port: 37647 } } }
I20260806 05:00:38.197409 30287 sys_catalog.cc:458] T 00000000000000000000000000000000 P dd7bd28b7a2e47c48fdbf033821161af [sys.catalog]: This master's current role is: FOLLOWER
I20260806 05:00:38.198138 30028 sys_catalog.cc:564] T 00000000000000000000000000000000 P dd7bd28b7a2e47c48fdbf033821161af [sys.catalog]: configured and running, proceeding with master startup.
W20260806 05:00:38.209434 30304 catalog_manager.cc:1558] T 00000000000000000000000000000000 P dd7bd28b7a2e47c48fdbf033821161af: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 05:00:38.209578 30304 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 05:00:38.207927 29920 master_runner.cc:186] Error getting master registration for 127.9.96.125:43181: OK, instance_id { permanent_uuid: "2f74eddebd414f74b7e76745204c5dad" instance_seqno: 1785992437514839 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260806 05:00:38.217247 29920 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260806 05:00:38.238381 29952 tablet.cc:2369] T 00000000000000000000000000000000 P dd7bd28b7a2e47c48fdbf033821161af: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 05:00:38.239840 30136 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.96.124" port: 37647 } attempt: 5
Aug 06 05:00:38 dist-test-slave-x4qh krb5kdc[29902](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992437, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.96.125@KRBTEST.COM for kudu/127.9.96.124@KRBTEST.COM
I20260806 05:00:38.258464 30138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:00:38.240405 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.124:37647 (local address 127.0.0.1:60654)
0806 05:00:38.240743 (+ 338us) negotiation.cc:107] Waiting for socket to connect
0806 05:00:38.240763 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0806 05:00:38.240874 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 05:00:38.241751 (+ 877us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 05:00:38.241756 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 05:00:38.242307 (+ 551us) client_negotiation.cc:189] Negotiated authn=SASL
0806 05:00:38.242524 (+ 217us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 05:00:38.242537 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 05:00:38.244473 (+ 1936us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 05:00:38.244480 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 05:00:38.245913 (+ 1433us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 05:00:38.245929 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 05:00:38.246933 (+ 1004us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 05:00:38.246939 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 05:00:38.247173 (+ 234us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:00:38.247894 (+ 721us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 05:00:38.247917 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0806 05:00:38.250789 (+ 2872us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 05:00:38.255138 (+ 4349us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 05:00:38.255143 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 05:00:38.255147 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 05:00:38.255791 (+ 644us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 05:00:38.256325 (+ 534us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 05:00:38.256330 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 05:00:38.256333 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 05:00:38.256568 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 05:00:38.257365 (+ 797us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 05:00:38.257371 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 05:00:38.257611 (+ 240us) client_negotiation.cc:769] Sending connection context
0806 05:00:38.257780 (+ 169us) client_negotiation.cc:240] Negotiation successful
0806 05:00:38.258081 (+ 301us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":212}
I20260806 05:00:38.258648 30266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:00:38.241080 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60654 (local address 127.9.96.124:37647)
0806 05:00:38.241272 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0806 05:00:38.241278 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 05:00:38.241295 (+ 17us) server_negotiation.cc:407] Connection header received
0806 05:00:38.241349 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 05:00:38.241354 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 05:00:38.241448 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 05:00:38.241556 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0806 05:00:38.242742 (+ 1186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:00:38.244309 (+ 1567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:00:38.246072 (+ 1763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:00:38.246769 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:00:38.246899 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:00:38.251013 (+ 4114us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 05:00:38.251022 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 05:00:38.251028 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 05:00:38.251075 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0806 05:00:38.254959 (+ 3884us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 05:00:38.255959 (+ 1000us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 05:00:38.255965 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 05:00:38.255969 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 05:00:38.256174 (+ 205us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 05:00:38.256702 (+ 528us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 05:00:38.256708 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 05:00:38.256711 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 05:00:38.257106 (+ 395us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 05:00:38.257297 (+ 191us) server_negotiation.cc:1035] Waiting for connection context
0806 05:00:38.257908 (+ 611us) server_negotiation.cc:299] Negotiation successful
0806 05:00:38.258246 (+ 338us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260806 05:00:38.264832 30136 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2f74eddebd414f74b7e76745204c5dad: Bootstrap starting.
I20260806 05:00:38.268878 30136 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2f74eddebd414f74b7e76745204c5dad: Neither blocks nor log segments found. Creating new log.
I20260806 05:00:38.270203 30136 log.cc:824] T 00000000000000000000000000000000 P 2f74eddebd414f74b7e76745204c5dad: Log is configured to *not* fsync() on all Append() calls
W20260806 05:00:38.273283 30179 tablet.cc:2369] T 00000000000000000000000000000000 P 76e710aebc494dfaa24f0ba592902b75: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 05:00:38.274225 30136 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2f74eddebd414f74b7e76745204c5dad: No bootstrap required, opened a new log
I20260806 05:00:38.285202 30136 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2f74eddebd414f74b7e76745204c5dad [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd7bd28b7a2e47c48fdbf033821161af" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 36525 } } peers { permanent_uuid: "2f74eddebd414f74b7e76745204c5dad" member_type: VOTER last_known_addr { host: "127.9.96.125" port: 43181 } } peers { permanent_uuid: "76e710aebc494dfaa24f0ba592902b75" member_type: VOTER last_known_addr { host: "127.9.96.124" port: 37647 } }
I20260806 05:00:38.285619 30136 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2f74eddebd414f74b7e76745204c5dad [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 05:00:38.285733 30136 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2f74eddebd414f74b7e76745204c5dad [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2f74eddebd414f74b7e76745204c5dad, State: Initialized, Role: FOLLOWER
I20260806 05:00:38.286374 30136 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2f74eddebd414f74b7e76745204c5dad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd7bd28b7a2e47c48fdbf033821161af" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 36525 } } peers { permanent_uuid: "2f74eddebd414f74b7e76745204c5dad" member_type: VOTER last_known_addr { host: "127.9.96.125" port: 43181 } } peers { permanent_uuid: "76e710aebc494dfaa24f0ba592902b75" member_type: VOTER last_known_addr { host: "127.9.96.124" port: 37647 } }
I20260806 05:00:38.287401 30311 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2f74eddebd414f74b7e76745204c5dad [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd7bd28b7a2e47c48fdbf033821161af" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 36525 } } peers { permanent_uuid: "2f74eddebd414f74b7e76745204c5dad" member_type: VOTER last_known_addr { host: "127.9.96.125" port: 43181 } } peers { permanent_uuid: "76e710aebc494dfaa24f0ba592902b75" member_type: VOTER last_known_addr { host: "127.9.96.124" port: 37647 } } }
I20260806 05:00:38.287902 30311 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2f74eddebd414f74b7e76745204c5dad [sys.catalog]: This master's current role is: FOLLOWER
I20260806 05:00:38.288692 30136 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2f74eddebd414f74b7e76745204c5dad [sys.catalog]: configured and running, proceeding with master startup.
I20260806 05:00:38.291191 30265 raft_consensus.cc:482] T 00000000000000000000000000000000 P 76e710aebc494dfaa24f0ba592902b75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 05:00:38.291580 30265 raft_consensus.cc:504] T 00000000000000000000000000000000 P 76e710aebc494dfaa24f0ba592902b75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd7bd28b7a2e47c48fdbf033821161af" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 36525 } } peers { permanent_uuid: "2f74eddebd414f74b7e76745204c5dad" member_type: VOTER last_known_addr { host: "127.9.96.125" port: 43181 } } peers { permanent_uuid: "76e710aebc494dfaa24f0ba592902b75" member_type: VOTER last_known_addr { host: "127.9.96.124" port: 37647 } }
I20260806 05:00:38.293095 30265 leader_election.cc:288] T 00000000000000000000000000000000 P 76e710aebc494dfaa24f0ba592902b75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers dd7bd28b7a2e47c48fdbf033821161af (127.9.96.126:36525), 2f74eddebd414f74b7e76745204c5dad (127.9.96.125:43181)
I20260806 05:00:38.296649 29987 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "76e710aebc494dfaa24f0ba592902b75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd7bd28b7a2e47c48fdbf033821161af" is_pre_election: true
I20260806 05:00:38.297253 29987 raft_consensus.cc:2454] T 00000000000000000000000000000000 P dd7bd28b7a2e47c48fdbf033821161af [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 76e710aebc494dfaa24f0ba592902b75 in term 0.
I20260806 05:00:38.297902 30100 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "76e710aebc494dfaa24f0ba592902b75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2f74eddebd414f74b7e76745204c5dad" is_pre_election: true
I20260806 05:00:38.298419 30100 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 2f74eddebd414f74b7e76745204c5dad [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 76e710aebc494dfaa24f0ba592902b75 in term 0.
I20260806 05:00:38.300522 30176 leader_election.cc:302] T 00000000000000000000000000000000 P 76e710aebc494dfaa24f0ba592902b75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2f74eddebd414f74b7e76745204c5dad, 76e710aebc494dfaa24f0ba592902b75; no voters:
I20260806 05:00:38.301788 30265 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 76e710aebc494dfaa24f0ba592902b75 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 05:00:38.301955 30265 raft_consensus.cc:482] T 00000000000000000000000000000000 P 76e710aebc494dfaa24f0ba592902b75 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 05:00:38.302055 30265 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 76e710aebc494dfaa24f0ba592902b75 [term 0 FOLLOWER]: Advancing to term 1
I20260806 05:00:38.305445 30265 raft_consensus.cc:504] T 00000000000000000000000000000000 P 76e710aebc494dfaa24f0ba592902b75 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd7bd28b7a2e47c48fdbf033821161af" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 36525 } } peers { permanent_uuid: "2f74eddebd414f74b7e76745204c5dad" member_type: VOTER last_known_addr { host: "127.9.96.125" port: 43181 } } peers { permanent_uuid: "76e710aebc494dfaa24f0ba592902b75" member_type: VOTER last_known_addr { host: "127.9.96.124" port: 37647 } }
I20260806 05:00:38.306584 29987 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "76e710aebc494dfaa24f0ba592902b75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd7bd28b7a2e47c48fdbf033821161af"
I20260806 05:00:38.306793 29987 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dd7bd28b7a2e47c48fdbf033821161af [term 0 FOLLOWER]: Advancing to term 1
I20260806 05:00:38.310279 29987 raft_consensus.cc:2454] T 00000000000000000000000000000000 P dd7bd28b7a2e47c48fdbf033821161af [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 76e710aebc494dfaa24f0ba592902b75 in term 1.
I20260806 05:00:38.310827 30176 leader_election.cc:302] T 00000000000000000000000000000000 P 76e710aebc494dfaa24f0ba592902b75 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 76e710aebc494dfaa24f0ba592902b75, dd7bd28b7a2e47c48fdbf033821161af; no voters:
I20260806 05:00:38.313292 30265 leader_election.cc:288] T 00000000000000000000000000000000 P 76e710aebc494dfaa24f0ba592902b75 [CANDIDATE]: Term 1 election: Requested vote from peers dd7bd28b7a2e47c48fdbf033821161af (127.9.96.126:36525), 2f74eddebd414f74b7e76745204c5dad (127.9.96.125:43181)
I20260806 05:00:38.313896 30265 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 76e710aebc494dfaa24f0ba592902b75 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 05:00:38.314272 30265 raft_consensus.cc:686] T 00000000000000000000000000000000 P 76e710aebc494dfaa24f0ba592902b75 [term 1 LEADER]: Becoming Leader. State: Replica: 76e710aebc494dfaa24f0ba592902b75, State: Running, Role: LEADER
I20260806 05:00:38.315462 30265 consensus_queue.cc:237] T 00000000000000000000000000000000 P 76e710aebc494dfaa24f0ba592902b75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd7bd28b7a2e47c48fdbf033821161af" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 36525 } } peers { permanent_uuid: "2f74eddebd414f74b7e76745204c5dad" member_type: VOTER last_known_addr { host: "127.9.96.125" port: 43181 } } peers { permanent_uuid: "76e710aebc494dfaa24f0ba592902b75" member_type: VOTER last_known_addr { host: "127.9.96.124" port: 37647 } }
I20260806 05:00:38.330437 30100 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "76e710aebc494dfaa24f0ba592902b75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2f74eddebd414f74b7e76745204c5dad"
I20260806 05:00:38.330721 30100 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2f74eddebd414f74b7e76745204c5dad [term 0 FOLLOWER]: Advancing to term 1
I20260806 05:00:38.334277 30100 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 2f74eddebd414f74b7e76745204c5dad [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 76e710aebc494dfaa24f0ba592902b75 in term 1.
I20260806 05:00:38.339887 30319 sys_catalog.cc:455] T 00000000000000000000000000000000 P 76e710aebc494dfaa24f0ba592902b75 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 76e710aebc494dfaa24f0ba592902b75. Latest consensus state: current_term: 1 leader_uuid: "76e710aebc494dfaa24f0ba592902b75" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd7bd28b7a2e47c48fdbf033821161af" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 36525 } } peers { permanent_uuid: "2f74eddebd414f74b7e76745204c5dad" member_type: VOTER last_known_addr { host: "127.9.96.125" port: 43181 } } peers { permanent_uuid: "76e710aebc494dfaa24f0ba592902b75" member_type: VOTER last_known_addr { host: "127.9.96.124" port: 37647 } } }
I20260806 05:00:38.340420 30319 sys_catalog.cc:458] T 00000000000000000000000000000000 P 76e710aebc494dfaa24f0ba592902b75 [sys.catalog]: This master's current role is: LEADER
I20260806 05:00:38.343989 30330 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260806 05:00:38.347491 30331 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 05:00:38.349870 30331 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 05:00:38.360960 30100 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 2f74eddebd414f74b7e76745204c5dad [term 1 FOLLOWER]: Refusing update from remote peer 76e710aebc494dfaa24f0ba592902b75: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 05:00:38.360988 29987 raft_consensus.cc:1261] T 00000000000000000000000000000000 P dd7bd28b7a2e47c48fdbf033821161af [term 1 FOLLOWER]: Refusing update from remote peer 76e710aebc494dfaa24f0ba592902b75: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 05:00:38.362253 30265 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 76e710aebc494dfaa24f0ba592902b75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd7bd28b7a2e47c48fdbf033821161af" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 36525 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 05:00:38.363696 30265 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 76e710aebc494dfaa24f0ba592902b75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2f74eddebd414f74b7e76745204c5dad" member_type: VOTER last_known_addr { host: "127.9.96.125" port: 43181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 05:00:38.388751 30287 sys_catalog.cc:455] T 00000000000000000000000000000000 P dd7bd28b7a2e47c48fdbf033821161af [sys.catalog]: SysCatalogTable state changed. Reason: New leader 76e710aebc494dfaa24f0ba592902b75. Latest consensus state: current_term: 1 leader_uuid: "76e710aebc494dfaa24f0ba592902b75" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd7bd28b7a2e47c48fdbf033821161af" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 36525 } } peers { permanent_uuid: "2f74eddebd414f74b7e76745204c5dad" member_type: VOTER last_known_addr { host: "127.9.96.125" port: 43181 } } peers { permanent_uuid: "76e710aebc494dfaa24f0ba592902b75" member_type: VOTER last_known_addr { host: "127.9.96.124" port: 37647 } } }
I20260806 05:00:38.389115 30287 sys_catalog.cc:458] T 00000000000000000000000000000000 P dd7bd28b7a2e47c48fdbf033821161af [sys.catalog]: This master's current role is: FOLLOWER
I20260806 05:00:38.389650 30311 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2f74eddebd414f74b7e76745204c5dad [sys.catalog]: SysCatalogTable state changed. Reason: New leader 76e710aebc494dfaa24f0ba592902b75. Latest consensus state: current_term: 1 leader_uuid: "76e710aebc494dfaa24f0ba592902b75" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd7bd28b7a2e47c48fdbf033821161af" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 36525 } } peers { permanent_uuid: "2f74eddebd414f74b7e76745204c5dad" member_type: VOTER last_known_addr { host: "127.9.96.125" port: 43181 } } peers { permanent_uuid: "76e710aebc494dfaa24f0ba592902b75" member_type: VOTER last_known_addr { host: "127.9.96.124" port: 37647 } } }
I20260806 05:00:38.389938 30311 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2f74eddebd414f74b7e76745204c5dad [sys.catalog]: This master's current role is: FOLLOWER
I20260806 05:00:38.393013 30319 sys_catalog.cc:455] T 00000000000000000000000000000000 P 76e710aebc494dfaa24f0ba592902b75 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "76e710aebc494dfaa24f0ba592902b75" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd7bd28b7a2e47c48fdbf033821161af" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 36525 } } peers { permanent_uuid: "2f74eddebd414f74b7e76745204c5dad" member_type: VOTER last_known_addr { host: "127.9.96.125" port: 43181 } } peers { permanent_uuid: "76e710aebc494dfaa24f0ba592902b75" member_type: VOTER last_known_addr { host: "127.9.96.124" port: 37647 } } }
I20260806 05:00:38.393311 30319 sys_catalog.cc:458] T 00000000000000000000000000000000 P 76e710aebc494dfaa24f0ba592902b75 [sys.catalog]: This master's current role is: LEADER
I20260806 05:00:38.395274 30331 catalog_manager.cc:1347] Generated new cluster ID: 4d6c8fe50b4642298a312360242ef7a7
I20260806 05:00:38.395491 30331 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 05:00:38.399410 30265 sys_catalog.cc:455] T 00000000000000000000000000000000 P 76e710aebc494dfaa24f0ba592902b75 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "76e710aebc494dfaa24f0ba592902b75" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd7bd28b7a2e47c48fdbf033821161af" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 36525 } } peers { permanent_uuid: "2f74eddebd414f74b7e76745204c5dad" member_type: VOTER last_known_addr { host: "127.9.96.125" port: 43181 } } peers { permanent_uuid: "76e710aebc494dfaa24f0ba592902b75" member_type: VOTER last_known_addr { host: "127.9.96.124" port: 37647 } } }
I20260806 05:00:38.399729 30265 sys_catalog.cc:458] T 00000000000000000000000000000000 P 76e710aebc494dfaa24f0ba592902b75 [sys.catalog]: This master's current role is: LEADER
I20260806 05:00:38.404325 30339 mvcc.cc:205] Tried to move back new op lower bound from 7315425027510398976 to 7315425027365027840. Current Snapshot: MvccSnapshot[applied={T|T < 7315425027510398976}]
I20260806 05:00:38.410465 30287 sys_catalog.cc:455] T 00000000000000000000000000000000 P dd7bd28b7a2e47c48fdbf033821161af [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "76e710aebc494dfaa24f0ba592902b75" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd7bd28b7a2e47c48fdbf033821161af" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 36525 } } peers { permanent_uuid: "2f74eddebd414f74b7e76745204c5dad" member_type: VOTER last_known_addr { host: "127.9.96.125" port: 43181 } } peers { permanent_uuid: "76e710aebc494dfaa24f0ba592902b75" member_type: VOTER last_known_addr { host: "127.9.96.124" port: 37647 } } }
I20260806 05:00:38.410727 30287 sys_catalog.cc:458] T 00000000000000000000000000000000 P dd7bd28b7a2e47c48fdbf033821161af [sys.catalog]: This master's current role is: FOLLOWER
I20260806 05:00:38.412976 30311 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2f74eddebd414f74b7e76745204c5dad [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "76e710aebc494dfaa24f0ba592902b75" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd7bd28b7a2e47c48fdbf033821161af" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 36525 } } peers { permanent_uuid: "2f74eddebd414f74b7e76745204c5dad" member_type: VOTER last_known_addr { host: "127.9.96.125" port: 43181 } } peers { permanent_uuid: "76e710aebc494dfaa24f0ba592902b75" member_type: VOTER last_known_addr { host: "127.9.96.124" port: 37647 } } }
I20260806 05:00:38.414824 30311 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2f74eddebd414f74b7e76745204c5dad [sys.catalog]: This master's current role is: FOLLOWER
I20260806 05:00:38.427922 30331 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 05:00:38.429932 30331 catalog_manager.cc:1504] Loading token signing keys...
I20260806 05:00:38.449060 30331 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 76e710aebc494dfaa24f0ba592902b75: Generated new TSK 0
I20260806 05:00:38.450047 30331 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 05:00:39.211709 30304 catalog_manager.cc:1259] Loaded cluster ID: 4d6c8fe50b4642298a312360242ef7a7
I20260806 05:00:39.211808 30304 catalog_manager.cc:1552] T 00000000000000000000000000000000 P dd7bd28b7a2e47c48fdbf033821161af: loading cluster ID for follower catalog manager: success
I20260806 05:00:39.214346 30304 catalog_manager.cc:1574] T 00000000000000000000000000000000 P dd7bd28b7a2e47c48fdbf033821161af: acquiring CA information for follower catalog manager: success
I20260806 05:00:39.216303 30304 catalog_manager.cc:1602] T 00000000000000000000000000000000 P dd7bd28b7a2e47c48fdbf033821161af: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 05:00:39.348294 30330 catalog_manager.cc:1259] Loaded cluster ID: 4d6c8fe50b4642298a312360242ef7a7
I20260806 05:00:39.348443 30330 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 2f74eddebd414f74b7e76745204c5dad: loading cluster ID for follower catalog manager: success
I20260806 05:00:39.351097 30330 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 2f74eddebd414f74b7e76745204c5dad: acquiring CA information for follower catalog manager: success
I20260806 05:00:39.352941 30330 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 2f74eddebd414f74b7e76745204c5dad: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 05:00:39.534482 9601 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu pid 29920
I20260806 05:00:39.535653 30018 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 05:00:39.644019 9601 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu with pid 29920
I20260806 05:00:39.679345 9601 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu pid 30037
I20260806 05:00:39.680555 30128 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 05:00:39.796358 9601 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu with pid 30037
I20260806 05:00:39.825838 9601 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu pid 30146
I20260806 05:00:39.826962 30240 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 05:00:39.936673 9601 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu with pid 30146
2026-08-06T05:00:39Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (4955 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785992227050689-9601-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 05:00:40 dist-test-slave-x4qh krb5kdc[30365](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 05:00:40 dist-test-slave-x4qh krb5kdc[30365](info): set up 2 sockets
Aug 06 05:00:40 dist-test-slave-x4qh krb5kdc[30365](info): commencing operation
krb5kdc: starting...
W20260806 05:00:41.254622 9601 mini_kdc.cc:121] Time spent starting KDC: real 1.268s user 0.003s sys 0.029s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785992227050689-9601-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785992227050689-9601-0/krb5kdc/test-user.keytab.
Aug 06 05:00:41 dist-test-slave-x4qh krb5kdc[30365](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992441, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T05:00:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T05:00:41Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.96.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.96.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.96.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.126.keytab.
Entry for principal kudu/127.9.96.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.126.keytab.
I20260806 05:00:41.515825 9601 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
/tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785992227050689-9601-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785992227050689-9601-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785992227050689-9601-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785992227050689-9601-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.96.126:41561
--webserver_interface=127.9.96.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.126.keytab
--principal=kudu/127.9.96.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.96.84:37721
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.96.126:41561
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785992227050689-9601-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785992227050689-9601-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785992227050689-9601-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 05:00:41.852630 30379 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 05:00:41.852975 30379 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 05:00:41.853150 30379 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 05:00:41.860208 30379 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 05:00:41.860338 30379 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 05:00:41.860442 30379 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 05:00:41.860554 30379 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 05:00:41.860642 30379 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 05:00:41.870347 30379 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.96.84:37721
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785992227050689-9601-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785992227050689-9601-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.96.126:41561
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.126.keytab
--principal=kudu/127.9.96.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.96.126:41561
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785992227050689-9601-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.96.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785992227050689-9601-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 03:56:12 UTC on ccycloud.quasar-guijbm.root.comops.site
build id 4569618
ASAN enabled
I20260806 05:00:41.871721 30379 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 05:00:41.873068 30379 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 05:00:41.885116 30388 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 05:00:41.885197 30387 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 05:00:41.885183 30390 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 05:00:41.886178 30379 server_base.cc:1034] running on GCE node
I20260806 05:00:41.886940 30379 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 05:00:41.888341 30379 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 05:00:41.889549 30379 hybrid_clock.cc:648] HybridClock initialized: now 1785992441889500 us; error 38 us; skew 500 ppm
I20260806 05:00:41.889917 30379 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 05:00:41 dist-test-slave-x4qh krb5kdc[30365](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992441, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.96.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 05:00:41.898188 30379 init.cc:377] Logged in from keytab as kudu/127.9.96.126@KRBTEST.COM (short username kudu)
I20260806 05:00:41.899757 30379 webserver.cc:460] Webserver started at http://127.9.96.126:36619/ using document root <none> and password file <none>
I20260806 05:00:41.900449 30379 fs_manager.cc:359] Metadata directory not provided
I20260806 05:00:41.900595 30379 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 05:00:41.900940 30379 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 05:00:41.903976 30379 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785992227050689-9601-0/minicluster-data/master-0/data/instance:
uuid: "80a774282412411b9889eaa09b25f91a"
format_stamp: "Formatted at 2026-08-06 05:00:41 on dist-test-slave-x4qh"
server_key: "fe12eb6346100adc2ab49803fef2c298"
server_key_iv: "333f4a53472eb309f487bab8a6b0fbf5"
server_key_version: "encryptionkey@0"
I20260806 05:00:41.904757 30379 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785992227050689-9601-0/minicluster-data/master-0/wal/instance:
uuid: "80a774282412411b9889eaa09b25f91a"
format_stamp: "Formatted at 2026-08-06 05:00:41 on dist-test-slave-x4qh"
server_key: "fe12eb6346100adc2ab49803fef2c298"
server_key_iv: "333f4a53472eb309f487bab8a6b0fbf5"
server_key_version: "encryptionkey@0"
I20260806 05:00:41.909730 30379 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260806 05:00:41.913012 30400 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 05:00:41.914242 30379 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260806 05:00:41.914430 30379 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785992227050689-9601-0/minicluster-data/master-0/data,/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785992227050689-9601-0/minicluster-data/master-0/wal
uuid: "80a774282412411b9889eaa09b25f91a"
format_stamp: "Formatted at 2026-08-06 05:00:41 on dist-test-slave-x4qh"
server_key: "fe12eb6346100adc2ab49803fef2c298"
server_key_iv: "333f4a53472eb309f487bab8a6b0fbf5"
server_key_version: "encryptionkey@0"
I20260806 05:00:41.914646 30379 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785992227050689-9601-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785992227050689-9601-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785992227050689-9601-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 05:00:41.944346 30379 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 05:00:41.948750 30379 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 05:00:41.949049 30379 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 05:00:41.968115 30379 rpc_server.cc:307] RPC server started. Bound to: 127.9.96.126:41561
I20260806 05:00:41.968151 30484 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.96.126:41561 every 8 connection(s)
I20260806 05:00:41.969429 30379 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785992227050689-9601-0/minicluster-data/master-0/data/info.pb
I20260806 05:00:41.973099 30487 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 05:00:41.976033 9601 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu as pid 30379
I20260806 05:00:41.976274 9601 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785992227050689-9601-0/minicluster-data/master-0/wal/instance
I20260806 05:00:41.977151 9601 external_mini_cluster.cc:1428] Setting key d438c1496c3a20f6009eb229d4d8e8b2
I20260806 05:00:41.985762 30487 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 80a774282412411b9889eaa09b25f91a: Bootstrap starting.
I20260806 05:00:41.989580 30487 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 80a774282412411b9889eaa09b25f91a: Neither blocks nor log segments found. Creating new log.
I20260806 05:00:41.990955 30487 log.cc:824] T 00000000000000000000000000000000 P 80a774282412411b9889eaa09b25f91a: Log is configured to *not* fsync() on all Append() calls
Aug 06 05:00:41 dist-test-slave-x4qh krb5kdc[30365](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992441, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.96.126@KRBTEST.COM
I20260806 05:00:41.994306 30487 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 80a774282412411b9889eaa09b25f91a: No bootstrap required, opened a new log
I20260806 05:00:42.000984 30493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:00:41.979037 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40932 (local address 127.9.96.126:41561)
0806 05:00:41.979895 (+ 858us) server_negotiation.cc:206] Beginning negotiation
0806 05:00:41.979905 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 05:00:41.979944 (+ 39us) server_negotiation.cc:407] Connection header received
0806 05:00:41.981598 (+ 1654us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 05:00:41.981618 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 05:00:41.981982 (+ 364us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 05:00:41.982289 (+ 307us) server_negotiation.cc:226] Negotiated authn=SASL
0806 05:00:41.984173 (+ 1884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:00:41.985523 (+ 1350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:00:41.987412 (+ 1889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:00:41.988080 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:00:41.988206 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:00:41.993603 (+ 5397us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 05:00:41.993609 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 05:00:41.993623 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 05:00:41.993659 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0806 05:00:41.996822 (+ 3163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 05:00:41.997696 (+ 874us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 05:00:41.997705 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 05:00:41.997711 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 05:00:41.997869 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 05:00:41.998327 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 05:00:41.998330 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 05:00:41.998332 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 05:00:41.998829 (+ 497us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 05:00:41.999049 (+ 220us) server_negotiation.cc:1035] Waiting for connection context
0806 05:00:41.999583 (+ 534us) server_negotiation.cc:299] Negotiation successful
0806 05:00:41.999954 (+ 371us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":661,"thread_start_us":452,"threads_started":1}
I20260806 05:00:42.003877 30487 raft_consensus.cc:348] T 00000000000000000000000000000000 P 80a774282412411b9889eaa09b25f91a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "80a774282412411b9889eaa09b25f91a" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 41561 } }
I20260806 05:00:42.004334 30487 raft_consensus.cc:374] T 00000000000000000000000000000000 P 80a774282412411b9889eaa09b25f91a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 05:00:42.004477 30487 raft_consensus.cc:729] T 00000000000000000000000000000000 P 80a774282412411b9889eaa09b25f91a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80a774282412411b9889eaa09b25f91a, State: Initialized, Role: FOLLOWER
I20260806 05:00:42.005024 9601 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 05:00:42.005096 30487 consensus_queue.cc:260] T 00000000000000000000000000000000 P 80a774282412411b9889eaa09b25f91a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "80a774282412411b9889eaa09b25f91a" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 41561 } }
I20260806 05:00:42.005371 30487 raft_consensus.cc:388] T 00000000000000000000000000000000 P 80a774282412411b9889eaa09b25f91a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 05:00:42.005527 30487 raft_consensus.cc:482] T 00000000000000000000000000000000 P 80a774282412411b9889eaa09b25f91a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 05:00:42.005709 30487 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 80a774282412411b9889eaa09b25f91a [term 0 FOLLOWER]: Advancing to term 1
I20260806 05:00:42.008044 30487 raft_consensus.cc:504] T 00000000000000000000000000000000 P 80a774282412411b9889eaa09b25f91a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "80a774282412411b9889eaa09b25f91a" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 41561 } }
I20260806 05:00:42.008615 30487 leader_election.cc:302] T 00000000000000000000000000000000 P 80a774282412411b9889eaa09b25f91a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 80a774282412411b9889eaa09b25f91a; no voters:
I20260806 05:00:42.009232 30487 leader_election.cc:288] T 00000000000000000000000000000000 P 80a774282412411b9889eaa09b25f91a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 05:00:42.009361 30496 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 80a774282412411b9889eaa09b25f91a [term 1 FOLLOWER]: Leader election won for term 1
I20260806 05:00:42.010079 30496 raft_consensus.cc:686] T 00000000000000000000000000000000 P 80a774282412411b9889eaa09b25f91a [term 1 LEADER]: Becoming Leader. State: Replica: 80a774282412411b9889eaa09b25f91a, State: Running, Role: LEADER
I20260806 05:00:42.010612 30496 consensus_queue.cc:237] T 00000000000000000000000000000000 P 80a774282412411b9889eaa09b25f91a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "80a774282412411b9889eaa09b25f91a" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 41561 } }
I20260806 05:00:42.011330 30487 sys_catalog.cc:564] T 00000000000000000000000000000000 P 80a774282412411b9889eaa09b25f91a [sys.catalog]: configured and running, proceeding with master startup.
I20260806 05:00:42.013738 30499 sys_catalog.cc:455] T 00000000000000000000000000000000 P 80a774282412411b9889eaa09b25f91a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 80a774282412411b9889eaa09b25f91a. Latest consensus state: current_term: 1 leader_uuid: "80a774282412411b9889eaa09b25f91a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "80a774282412411b9889eaa09b25f91a" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 41561 } } }
I20260806 05:00:42.014219 30499 sys_catalog.cc:458] T 00000000000000000000000000000000 P 80a774282412411b9889eaa09b25f91a [sys.catalog]: This master's current role is: LEADER
I20260806 05:00:42.018057 30497 sys_catalog.cc:455] T 00000000000000000000000000000000 P 80a774282412411b9889eaa09b25f91a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "80a774282412411b9889eaa09b25f91a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "80a774282412411b9889eaa09b25f91a" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 41561 } } }
I20260806 05:00:42.018313 30497 sys_catalog.cc:458] T 00000000000000000000000000000000 P 80a774282412411b9889eaa09b25f91a [sys.catalog]: This master's current role is: LEADER
I20260806 05:00:42.019336 30508 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 05:00:42.023345 30508 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 05:00:42.031293 30508 catalog_manager.cc:1347] Generated new cluster ID: e2fb4836cb8a457a8847e2816f09a08a
I20260806 05:00:42.031430 30508 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 05:00:42.075011 30508 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 05:00:42.076478 30508 catalog_manager.cc:1504] Loading token signing keys...
I20260806 05:00:42.085983 30508 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 80a774282412411b9889eaa09b25f91a: Generated new TSK 0
I20260806 05:00:42.086838 30508 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 05:00:42.117853 9601 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu pid 30379
I20260806 05:00:42.119155 30480 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 05:00:42.214054 9601 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu with pid 30379
2026-08-06T05:00:42Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (2272 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785992227050689-9601-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 05:00:42 dist-test-slave-x4qh krb5kdc[30534](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 05:00:42 dist-test-slave-x4qh krb5kdc[30534](info): set up 2 sockets
Aug 06 05:00:42 dist-test-slave-x4qh krb5kdc[30534](info): commencing operation
krb5kdc: starting...
W20260806 05:00:43.516004 9601 mini_kdc.cc:121] Time spent starting KDC: real 1.257s user 0.003s sys 0.030s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785992227050689-9601-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785992227050689-9601-0/krb5kdc/test-user.keytab.
Aug 06 05:00:43 dist-test-slave-x4qh krb5kdc[30534](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992443, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T05:00:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T05:00:43Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.96.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.96.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.96.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.126.keytab.
Entry for principal kudu/127.9.96.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.126.keytab.
I20260806 05:00:43.770031 9601 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
/tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785992227050689-9601-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785992227050689-9601-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785992227050689-9601-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785992227050689-9601-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.96.126:39911
--webserver_interface=127.9.96.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.126.keytab
--principal=kudu/127.9.96.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.96.84:40741
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.96.126:39911
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785992227050689-9601-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785992227050689-9601-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785992227050689-9601-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 05:00:44.100454 30555 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 05:00:44.100785 30555 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 05:00:44.100958 30555 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 05:00:44.107973 30555 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 05:00:44.108067 30555 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 05:00:44.108117 30555 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 05:00:44.108173 30555 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 05:00:44.108217 30555 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 05:00:44.117978 30555 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.96.84:40741
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785992227050689-9601-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785992227050689-9601-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.96.126:39911
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.126.keytab
--principal=kudu/127.9.96.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.96.126:39911
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785992227050689-9601-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.96.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785992227050689-9601-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 03:56:12 UTC on ccycloud.quasar-guijbm.root.comops.site
build id 4569618
ASAN enabled
I20260806 05:00:44.119135 30555 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 05:00:44.120364 30555 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 05:00:44.132306 30563 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 05:00:44.132339 30564 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 05:00:44.132463 30567 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 05:00:44.133164 30555 server_base.cc:1034] running on GCE node
I20260806 05:00:44.133919 30555 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 05:00:44.135170 30555 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 05:00:44.136448 30555 hybrid_clock.cc:648] HybridClock initialized: now 1785992444136389 us; error 46 us; skew 500 ppm
I20260806 05:00:44.136799 30555 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 05:00:44 dist-test-slave-x4qh krb5kdc[30534](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992444, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.96.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 05:00:44.139850 30555 init.cc:377] Logged in from keytab as kudu/127.9.96.126@KRBTEST.COM (short username kudu)
I20260806 05:00:44.141203 30555 webserver.cc:460] Webserver started at http://127.9.96.126:40163/ using document root <none> and password file <none>
I20260806 05:00:44.141822 30555 fs_manager.cc:359] Metadata directory not provided
I20260806 05:00:44.141940 30555 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 05:00:44.142180 30555 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 05:00:44.145109 30555 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785992227050689-9601-0/minicluster-data/master-0/data/instance:
uuid: "245b5b6f6eca4e65b1c4e9298645efec"
format_stamp: "Formatted at 2026-08-06 05:00:44 on dist-test-slave-x4qh"
server_key: "2aded742f873db7f2e686dc6a545a650"
server_key_iv: "0de062344079dc557791a2862f33c17a"
server_key_version: "encryptionkey@0"
I20260806 05:00:44.145833 30555 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785992227050689-9601-0/minicluster-data/master-0/wal/instance:
uuid: "245b5b6f6eca4e65b1c4e9298645efec"
format_stamp: "Formatted at 2026-08-06 05:00:44 on dist-test-slave-x4qh"
server_key: "2aded742f873db7f2e686dc6a545a650"
server_key_iv: "0de062344079dc557791a2862f33c17a"
server_key_version: "encryptionkey@0"
I20260806 05:00:44.150946 30555 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260806 05:00:44.154286 30578 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 05:00:44.155679 30555 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 05:00:44.155862 30555 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785992227050689-9601-0/minicluster-data/master-0/data,/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785992227050689-9601-0/minicluster-data/master-0/wal
uuid: "245b5b6f6eca4e65b1c4e9298645efec"
format_stamp: "Formatted at 2026-08-06 05:00:44 on dist-test-slave-x4qh"
server_key: "2aded742f873db7f2e686dc6a545a650"
server_key_iv: "0de062344079dc557791a2862f33c17a"
server_key_version: "encryptionkey@0"
I20260806 05:00:44.156061 30555 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785992227050689-9601-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785992227050689-9601-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785992227050689-9601-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 05:00:44.180027 30555 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 05:00:44.183893 30555 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 05:00:44.184183 30555 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 05:00:44.203523 30555 rpc_server.cc:307] RPC server started. Bound to: 127.9.96.126:39911
I20260806 05:00:44.203536 30656 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.96.126:39911 every 8 connection(s)
I20260806 05:00:44.204905 30555 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785992227050689-9601-0/minicluster-data/master-0/data/info.pb
I20260806 05:00:44.208557 30658 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 05:00:44.212852 9601 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu as pid 30555
I20260806 05:00:44.213085 9601 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785992227050689-9601-0/minicluster-data/master-0/wal/instance
I20260806 05:00:44.213753 9601 external_mini_cluster.cc:1428] Setting key 00f4fd68d259f155044247ec8f6f8c7a
I20260806 05:00:44.219549 30658 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 245b5b6f6eca4e65b1c4e9298645efec: Bootstrap starting.
I20260806 05:00:44.223111 30658 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 245b5b6f6eca4e65b1c4e9298645efec: Neither blocks nor log segments found. Creating new log.
I20260806 05:00:44.224426 30658 log.cc:824] T 00000000000000000000000000000000 P 245b5b6f6eca4e65b1c4e9298645efec: Log is configured to *not* fsync() on all Append() calls
I20260806 05:00:44.227910 30658 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 245b5b6f6eca4e65b1c4e9298645efec: No bootstrap required, opened a new log
Aug 06 05:00:44 dist-test-slave-x4qh krb5kdc[30534](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992443, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.96.126@KRBTEST.COM
I20260806 05:00:44.238778 30666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:00:44.215760 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48770 (local address 127.9.96.126:39911)
0806 05:00:44.216613 (+ 853us) server_negotiation.cc:206] Beginning negotiation
0806 05:00:44.216627 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0806 05:00:44.216672 (+ 45us) server_negotiation.cc:407] Connection header received
0806 05:00:44.218414 (+ 1742us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 05:00:44.218435 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 05:00:44.218781 (+ 346us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 05:00:44.219076 (+ 295us) server_negotiation.cc:226] Negotiated authn=SASL
0806 05:00:44.221103 (+ 2027us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:00:44.222433 (+ 1330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:00:44.224414 (+ 1981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:00:44.225343 (+ 929us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:00:44.225585 (+ 242us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:00:44.230296 (+ 4711us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 05:00:44.230307 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 05:00:44.230325 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 05:00:44.230369 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0806 05:00:44.234413 (+ 4044us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 05:00:44.235112 (+ 699us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 05:00:44.235120 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 05:00:44.235128 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0806 05:00:44.235338 (+ 210us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 05:00:44.235721 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 05:00:44.235726 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 05:00:44.235730 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 05:00:44.236441 (+ 711us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 05:00:44.236773 (+ 332us) server_negotiation.cc:1035] Waiting for connection context
0806 05:00:44.237090 (+ 317us) server_negotiation.cc:299] Negotiation successful
0806 05:00:44.237514 (+ 424us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":661,"thread_start_us":416,"threads_started":1}
I20260806 05:00:44.238579 30658 raft_consensus.cc:348] T 00000000000000000000000000000000 P 245b5b6f6eca4e65b1c4e9298645efec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "245b5b6f6eca4e65b1c4e9298645efec" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 39911 } }
I20260806 05:00:44.239028 30658 raft_consensus.cc:374] T 00000000000000000000000000000000 P 245b5b6f6eca4e65b1c4e9298645efec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 05:00:44.239125 30658 raft_consensus.cc:729] T 00000000000000000000000000000000 P 245b5b6f6eca4e65b1c4e9298645efec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 245b5b6f6eca4e65b1c4e9298645efec, State: Initialized, Role: FOLLOWER
I20260806 05:00:44.239768 30658 consensus_queue.cc:260] T 00000000000000000000000000000000 P 245b5b6f6eca4e65b1c4e9298645efec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "245b5b6f6eca4e65b1c4e9298645efec" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 39911 } }
I20260806 05:00:44.240013 30658 raft_consensus.cc:388] T 00000000000000000000000000000000 P 245b5b6f6eca4e65b1c4e9298645efec [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 05:00:44.240177 30658 raft_consensus.cc:482] T 00000000000000000000000000000000 P 245b5b6f6eca4e65b1c4e9298645efec [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 05:00:44.240391 30658 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 245b5b6f6eca4e65b1c4e9298645efec [term 0 FOLLOWER]: Advancing to term 1
I20260806 05:00:44.242628 30658 raft_consensus.cc:504] T 00000000000000000000000000000000 P 245b5b6f6eca4e65b1c4e9298645efec [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "245b5b6f6eca4e65b1c4e9298645efec" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 39911 } }
I20260806 05:00:44.243113 30658 leader_election.cc:302] T 00000000000000000000000000000000 P 245b5b6f6eca4e65b1c4e9298645efec [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 245b5b6f6eca4e65b1c4e9298645efec; no voters:
I20260806 05:00:44.243842 30658 leader_election.cc:288] T 00000000000000000000000000000000 P 245b5b6f6eca4e65b1c4e9298645efec [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 05:00:44.243973 30668 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 245b5b6f6eca4e65b1c4e9298645efec [term 1 FOLLOWER]: Leader election won for term 1
I20260806 05:00:44.244738 30668 raft_consensus.cc:686] T 00000000000000000000000000000000 P 245b5b6f6eca4e65b1c4e9298645efec [term 1 LEADER]: Becoming Leader. State: Replica: 245b5b6f6eca4e65b1c4e9298645efec, State: Running, Role: LEADER
I20260806 05:00:44.245352 30668 consensus_queue.cc:237] T 00000000000000000000000000000000 P 245b5b6f6eca4e65b1c4e9298645efec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "245b5b6f6eca4e65b1c4e9298645efec" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 39911 } }
I20260806 05:00:44.246101 30658 sys_catalog.cc:564] T 00000000000000000000000000000000 P 245b5b6f6eca4e65b1c4e9298645efec [sys.catalog]: configured and running, proceeding with master startup.
I20260806 05:00:44.250643 30670 sys_catalog.cc:455] T 00000000000000000000000000000000 P 245b5b6f6eca4e65b1c4e9298645efec [sys.catalog]: SysCatalogTable state changed. Reason: New leader 245b5b6f6eca4e65b1c4e9298645efec. Latest consensus state: current_term: 1 leader_uuid: "245b5b6f6eca4e65b1c4e9298645efec" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "245b5b6f6eca4e65b1c4e9298645efec" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 39911 } } }
I20260806 05:00:44.251065 30670 sys_catalog.cc:458] T 00000000000000000000000000000000 P 245b5b6f6eca4e65b1c4e9298645efec [sys.catalog]: This master's current role is: LEADER
I20260806 05:00:44.254562 30669 sys_catalog.cc:455] T 00000000000000000000000000000000 P 245b5b6f6eca4e65b1c4e9298645efec [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "245b5b6f6eca4e65b1c4e9298645efec" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "245b5b6f6eca4e65b1c4e9298645efec" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 39911 } } }
I20260806 05:00:44.254937 30669 sys_catalog.cc:458] T 00000000000000000000000000000000 P 245b5b6f6eca4e65b1c4e9298645efec [sys.catalog]: This master's current role is: LEADER
I20260806 05:00:44.255602 30684 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 05:00:44.261119 30684 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 05:00:44.269975 30684 catalog_manager.cc:1347] Generated new cluster ID: c694851103804b2c91a29a7c087b7b2a
I20260806 05:00:44.270084 30684 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 05:00:44.282713 30684 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 05:00:44.284055 30684 catalog_manager.cc:1504] Loading token signing keys...
I20260806 05:00:44.303056 30684 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 245b5b6f6eca4e65b1c4e9298645efec: Generated new TSK 0
I20260806 05:00:44.303957 30684 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.9.96.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.96.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.96.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.65.keytab.
Entry for principal kudu/127.9.96.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.65.keytab.
I20260806 05:00:44.382929 9601 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
/tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785992227050689-9601-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785992227050689-9601-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785992227050689-9601-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785992227050689-9601-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.96.65:0
--local_ip_for_outbound_sockets=127.9.96.65
--webserver_interface=127.9.96.65
--webserver_port=0
--tserver_master_addrs=127.9.96.126:39911
--keytab_file=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.65.keytab
--principal=kudu/127.9.96.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.96.84:40741
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785992227050689-9601-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785992227050689-9601-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785992227050689-9601-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 05:00:44.729738 30707 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 05:00:44.729993 30707 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 05:00:44.730087 30707 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 05:00:44.737473 30707 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 05:00:44.737620 30707 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 05:00:44.737789 30707 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.96.65
I20260806 05:00:44.750056 30707 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.96.84:40741
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785992227050689-9601-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785992227050689-9601-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.65.keytab
--principal=kudu/127.9.96.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.96.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785992227050689-9601-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.96.65
--webserver_port=0
--tserver_master_addrs=127.9.96.126:39911
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.96.65
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785992227050689-9601-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 03:56:12 UTC on ccycloud.quasar-guijbm.root.comops.site
build id 4569618
ASAN enabled
I20260806 05:00:44.751343 30707 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 05:00:44.752746 30707 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 05:00:44.766100 30715 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 05:00:44.766134 30716 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 05:00:44.766184 30718 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 05:00:44.766981 30707 server_base.cc:1034] running on GCE node
I20260806 05:00:44.767638 30707 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 05:00:44.768519 30707 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 05:00:44.769745 30707 hybrid_clock.cc:648] HybridClock initialized: now 1785992444769716 us; error 44 us; skew 500 ppm
I20260806 05:00:44.770051 30707 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 05:00:44 dist-test-slave-x4qh krb5kdc[30534](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992444, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.96.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 05:00:44.773196 30707 init.cc:377] Logged in from keytab as kudu/127.9.96.65@KRBTEST.COM (short username kudu)
I20260806 05:00:44.774608 30707 webserver.cc:460] Webserver started at http://127.9.96.65:43473/ using document root <none> and password file <none>
I20260806 05:00:44.775272 30707 fs_manager.cc:359] Metadata directory not provided
I20260806 05:00:44.775449 30707 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 05:00:44.775759 30707 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 05:00:44.778599 30707 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785992227050689-9601-0/minicluster-data/ts-0/data/instance:
uuid: "7bf95bbc6b8240e4b043b56c9cf673dc"
format_stamp: "Formatted at 2026-08-06 05:00:44 on dist-test-slave-x4qh"
server_key: "efa1633140f1902db8c8b3c77b52919e"
server_key_iv: "e525cdafc2ca69de881c87c31504b8ae"
server_key_version: "encryptionkey@0"
I20260806 05:00:44.779418 30707 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785992227050689-9601-0/minicluster-data/ts-0/wal/instance:
uuid: "7bf95bbc6b8240e4b043b56c9cf673dc"
format_stamp: "Formatted at 2026-08-06 05:00:44 on dist-test-slave-x4qh"
server_key: "efa1633140f1902db8c8b3c77b52919e"
server_key_iv: "e525cdafc2ca69de881c87c31504b8ae"
server_key_version: "encryptionkey@0"
I20260806 05:00:44.784546 30707 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.002s
I20260806 05:00:44.787869 30730 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 05:00:44.789054 30707 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260806 05:00:44.789223 30707 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785992227050689-9601-0/minicluster-data/ts-0/data,/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785992227050689-9601-0/minicluster-data/ts-0/wal
uuid: "7bf95bbc6b8240e4b043b56c9cf673dc"
format_stamp: "Formatted at 2026-08-06 05:00:44 on dist-test-slave-x4qh"
server_key: "efa1633140f1902db8c8b3c77b52919e"
server_key_iv: "e525cdafc2ca69de881c87c31504b8ae"
server_key_version: "encryptionkey@0"
I20260806 05:00:44.789426 30707 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785992227050689-9601-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785992227050689-9601-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785992227050689-9601-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 05:00:44.804926 30707 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 05:00:44.808770 30707 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 05:00:44.809070 30707 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 05:00:44.809985 30707 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 05:00:44.811311 30707 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 05:00:44.811442 30707 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.001s
I20260806 05:00:44.811601 30707 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 05:00:44.811677 30707 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 05:00:44.845901 30707 rpc_server.cc:307] RPC server started. Bound to: 127.9.96.65:38017
I20260806 05:00:44.845932 30903 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.96.65:38017 every 8 connection(s)
I20260806 05:00:44.847322 30707 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785992227050689-9601-0/minicluster-data/ts-0/data/info.pb
I20260806 05:00:44.853524 9601 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu as pid 30707
I20260806 05:00:44.853744 9601 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785992227050689-9601-0/minicluster-data/ts-0/wal/instance
I20260806 05:00:44.854380 9601 external_mini_cluster.cc:1428] Setting key c58b491b6adbba0792e299ed5178bbb4
Aug 06 05:00:44 dist-test-slave-x4qh krb5kdc[30534](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992444, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.96.65@KRBTEST.COM for kudu/127.9.96.126@KRBTEST.COM
I20260806 05:00:44.868436 30909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:00:44.849842 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.96.65:58307 (local address 127.9.96.126:39911)
0806 05:00:44.850317 (+ 475us) server_negotiation.cc:206] Beginning negotiation
0806 05:00:44.850322 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 05:00:44.851110 (+ 788us) server_negotiation.cc:407] Connection header received
0806 05:00:44.852970 (+ 1860us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 05:00:44.852975 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 05:00:44.853072 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 05:00:44.853192 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0806 05:00:44.854823 (+ 1631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:00:44.855792 (+ 969us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:00:44.857474 (+ 1682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:00:44.857868 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:00:44.858013 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:00:44.862220 (+ 4207us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 05:00:44.862225 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 05:00:44.862228 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 05:00:44.862261 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0806 05:00:44.864847 (+ 2586us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 05:00:44.865808 (+ 961us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 05:00:44.865815 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 05:00:44.865820 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 05:00:44.865963 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 05:00:44.866460 (+ 497us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 05:00:44.866467 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 05:00:44.866472 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 05:00:44.866795 (+ 323us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 05:00:44.866933 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0806 05:00:44.867824 (+ 891us) server_negotiation.cc:299] Negotiation successful
0806 05:00:44.868050 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":366,"thread_start_us":261,"threads_started":1}
I20260806 05:00:44.869345 30911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:00:44.850251 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:39911 (local address 127.9.96.65:58307)
0806 05:00:44.850948 (+ 697us) negotiation.cc:107] Waiting for socket to connect
0806 05:00:44.850992 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0806 05:00:44.852695 (+ 1703us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 05:00:44.853448 (+ 753us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 05:00:44.853455 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 05:00:44.854039 (+ 584us) client_negotiation.cc:189] Negotiated authn=SASL
0806 05:00:44.854650 (+ 611us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 05:00:44.854669 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 05:00:44.855988 (+ 1319us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 05:00:44.855992 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 05:00:44.857289 (+ 1297us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 05:00:44.857301 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 05:00:44.858101 (+ 800us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 05:00:44.858110 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 05:00:44.858351 (+ 241us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:00:44.859052 (+ 701us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 05:00:44.859075 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0806 05:00:44.862009 (+ 2934us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 05:00:44.865007 (+ 2998us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 05:00:44.865020 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 05:00:44.865038 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0806 05:00:44.865686 (+ 648us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 05:00:44.866080 (+ 394us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 05:00:44.866087 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 05:00:44.866093 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0806 05:00:44.866350 (+ 257us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 05:00:44.866933 (+ 583us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 05:00:44.866943 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 05:00:44.867432 (+ 489us) client_negotiation.cc:769] Sending connection context
0806 05:00:44.867721 (+ 289us) client_negotiation.cc:240] Negotiation successful
0806 05:00:44.868240 (+ 519us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":467,"thread_start_us":267,"threads_started":1}
I20260806 05:00:44.870893 30906 heartbeater.cc:347] Connected to a master server at 127.9.96.126:39911
I20260806 05:00:44.871250 30906 heartbeater.cc:464] Registering TS with master...
I20260806 05:00:44.872177 30906 heartbeater.cc:511] Master 127.9.96.126:39911 requested a full tablet report, sending...
I20260806 05:00:44.874399 30607 ts_manager.cc:194] Registered new tserver with Master: 7bf95bbc6b8240e4b043b56c9cf673dc (127.9.96.65:38017)
I20260806 05:00:44.875897 30607 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.96.65@KRBTEST.COM'} at 127.9.96.65:58307
WARNING: no policy specified for kudu/127.9.96.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.96.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.96.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.66.keytab.
Entry for principal kudu/127.9.96.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.66.keytab.
I20260806 05:00:44.918581 9601 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
/tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785992227050689-9601-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785992227050689-9601-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785992227050689-9601-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785992227050689-9601-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.96.66:0
--local_ip_for_outbound_sockets=127.9.96.66
--webserver_interface=127.9.96.66
--webserver_port=0
--tserver_master_addrs=127.9.96.126:39911
--keytab_file=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.66.keytab
--principal=kudu/127.9.96.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.96.84:40741
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785992227050689-9601-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785992227050689-9601-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785992227050689-9601-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 05:00:45.252190 30916 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 05:00:45.252456 30916 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 05:00:45.252632 30916 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 05:00:45.260056 30916 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 05:00:45.260195 30916 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 05:00:45.260375 30916 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.96.66
I20260806 05:00:45.270026 30916 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.96.84:40741
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785992227050689-9601-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785992227050689-9601-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.66.keytab
--principal=kudu/127.9.96.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.96.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785992227050689-9601-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.96.66
--webserver_port=0
--tserver_master_addrs=127.9.96.126:39911
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.96.66
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785992227050689-9601-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 03:56:12 UTC on ccycloud.quasar-guijbm.root.comops.site
build id 4569618
ASAN enabled
I20260806 05:00:45.271214 30916 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 05:00:45.272522 30916 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 05:00:45.285740 30927 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 05:00:45.285849 30925 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 05:00:45.285820 30923 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 05:00:45.287047 30916 server_base.cc:1034] running on GCE node
I20260806 05:00:45.287705 30916 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 05:00:45.288589 30916 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 05:00:45.289822 30916 hybrid_clock.cc:648] HybridClock initialized: now 1785992445289773 us; error 51 us; skew 500 ppm
I20260806 05:00:45.290128 30916 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 05:00:45 dist-test-slave-x4qh krb5kdc[30534](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992445, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.96.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 05:00:45.293448 30916 init.cc:377] Logged in from keytab as kudu/127.9.96.66@KRBTEST.COM (short username kudu)
I20260806 05:00:45.294945 30916 webserver.cc:460] Webserver started at http://127.9.96.66:45935/ using document root <none> and password file <none>
I20260806 05:00:45.295811 30916 fs_manager.cc:359] Metadata directory not provided
I20260806 05:00:45.295959 30916 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 05:00:45.296258 30916 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 05:00:45.299060 30916 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785992227050689-9601-0/minicluster-data/ts-1/data/instance:
uuid: "2f286e3a16b847239e12defeff122bec"
format_stamp: "Formatted at 2026-08-06 05:00:45 on dist-test-slave-x4qh"
server_key: "5b9483aab78a2362524a22fd7fa7400b"
server_key_iv: "d6f1132ffd9b9249b72b90a77a248723"
server_key_version: "encryptionkey@0"
I20260806 05:00:45.299834 30916 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785992227050689-9601-0/minicluster-data/ts-1/wal/instance:
uuid: "2f286e3a16b847239e12defeff122bec"
format_stamp: "Formatted at 2026-08-06 05:00:45 on dist-test-slave-x4qh"
server_key: "5b9483aab78a2362524a22fd7fa7400b"
server_key_iv: "d6f1132ffd9b9249b72b90a77a248723"
server_key_version: "encryptionkey@0"
I20260806 05:00:45.305084 30916 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260806 05:00:45.308416 30937 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 05:00:45.309655 30916 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260806 05:00:45.309834 30916 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785992227050689-9601-0/minicluster-data/ts-1/data,/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785992227050689-9601-0/minicluster-data/ts-1/wal
uuid: "2f286e3a16b847239e12defeff122bec"
format_stamp: "Formatted at 2026-08-06 05:00:45 on dist-test-slave-x4qh"
server_key: "5b9483aab78a2362524a22fd7fa7400b"
server_key_iv: "d6f1132ffd9b9249b72b90a77a248723"
server_key_version: "encryptionkey@0"
I20260806 05:00:45.310034 30916 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785992227050689-9601-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785992227050689-9601-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785992227050689-9601-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 05:00:45.333532 30916 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 05:00:45.337615 30916 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 05:00:45.337919 30916 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 05:00:45.338899 30916 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 05:00:45.340283 30916 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 05:00:45.340394 30916 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 05:00:45.340545 30916 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 05:00:45.340617 30916 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 05:00:45.374806 30916 rpc_server.cc:307] RPC server started. Bound to: 127.9.96.66:43613
I20260806 05:00:45.374899 31107 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.96.66:43613 every 8 connection(s)
I20260806 05:00:45.376199 30916 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785992227050689-9601-0/minicluster-data/ts-1/data/info.pb
I20260806 05:00:45.385673 9601 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu as pid 30916
I20260806 05:00:45.385870 9601 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785992227050689-9601-0/minicluster-data/ts-1/wal/instance
I20260806 05:00:45.386435 9601 external_mini_cluster.cc:1428] Setting key 71bea9809da00948786008d7558d6a21
Aug 06 05:00:45 dist-test-slave-x4qh krb5kdc[30534](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992445, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.96.66@KRBTEST.COM for kudu/127.9.96.126@KRBTEST.COM
I20260806 05:00:45.397888 31110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:00:45.378754 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.96.66:43079 (local address 127.9.96.126:39911)
0806 05:00:45.379185 (+ 431us) server_negotiation.cc:206] Beginning negotiation
0806 05:00:45.379189 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 05:00:45.380217 (+ 1028us) server_negotiation.cc:407] Connection header received
0806 05:00:45.381992 (+ 1775us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 05:00:45.381996 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 05:00:45.382070 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 05:00:45.382169 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0806 05:00:45.383641 (+ 1472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:00:45.384351 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:00:45.386200 (+ 1849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:00:45.386656 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:00:45.386751 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:00:45.390646 (+ 3895us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 05:00:45.390656 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 05:00:45.390665 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 05:00:45.390714 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0806 05:00:45.394485 (+ 3771us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 05:00:45.395221 (+ 736us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 05:00:45.395229 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 05:00:45.395235 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 05:00:45.395424 (+ 189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 05:00:45.395880 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 05:00:45.395887 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 05:00:45.395893 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 05:00:45.396287 (+ 394us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 05:00:45.396442 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0806 05:00:45.397118 (+ 676us) server_negotiation.cc:299] Negotiation successful
0806 05:00:45.397372 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":322,"thread_start_us":220,"threads_started":1}
I20260806 05:00:45.398480 31112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:00:45.379292 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:39911 (local address 127.9.96.66:43079)
0806 05:00:45.380052 (+ 760us) negotiation.cc:107] Waiting for socket to connect
0806 05:00:45.380088 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0806 05:00:45.381724 (+ 1636us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 05:00:45.382364 (+ 640us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 05:00:45.382371 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 05:00:45.382869 (+ 498us) client_negotiation.cc:189] Negotiated authn=SASL
0806 05:00:45.383482 (+ 613us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 05:00:45.383500 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 05:00:45.384493 (+ 993us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 05:00:45.384496 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 05:00:45.386062 (+ 1566us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 05:00:45.386076 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 05:00:45.386893 (+ 817us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 05:00:45.386901 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 05:00:45.387171 (+ 270us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:00:45.387988 (+ 817us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 05:00:45.388023 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0806 05:00:45.390401 (+ 2378us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 05:00:45.394619 (+ 4218us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 05:00:45.394626 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 05:00:45.394640 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0806 05:00:45.395079 (+ 439us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 05:00:45.395550 (+ 471us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 05:00:45.395554 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 05:00:45.395556 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 05:00:45.395751 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 05:00:45.396417 (+ 666us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 05:00:45.396423 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 05:00:45.396725 (+ 302us) client_negotiation.cc:769] Sending connection context
0806 05:00:45.396959 (+ 234us) client_negotiation.cc:240] Negotiation successful
0806 05:00:45.397349 (+ 390us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":541,"thread_start_us":268,"threads_started":1}
I20260806 05:00:45.399966 31108 heartbeater.cc:347] Connected to a master server at 127.9.96.126:39911
I20260806 05:00:45.400332 31108 heartbeater.cc:464] Registering TS with master...
I20260806 05:00:45.401221 31108 heartbeater.cc:511] Master 127.9.96.126:39911 requested a full tablet report, sending...
I20260806 05:00:45.402769 30607 ts_manager.cc:194] Registered new tserver with Master: 2f286e3a16b847239e12defeff122bec (127.9.96.66:43613)
I20260806 05:00:45.403650 30607 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.96.66@KRBTEST.COM'} at 127.9.96.66:43079
WARNING: no policy specified for kudu/127.9.96.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.96.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.96.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.67.keytab.
Entry for principal kudu/127.9.96.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.67.keytab.
I20260806 05:00:45.448112 9601 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
/tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785992227050689-9601-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785992227050689-9601-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785992227050689-9601-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785992227050689-9601-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.96.67:0
--local_ip_for_outbound_sockets=127.9.96.67
--webserver_interface=127.9.96.67
--webserver_port=0
--tserver_master_addrs=127.9.96.126:39911
--keytab_file=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.67.keytab
--principal=kudu/127.9.96.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.96.84:40741
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785992227050689-9601-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785992227050689-9601-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785992227050689-9601-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 05:00:45.787949 31117 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 05:00:45.788247 31117 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 05:00:45.788424 31117 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 05:00:45.795624 31117 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 05:00:45.795758 31117 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 05:00:45.795935 31117 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.96.67
I20260806 05:00:45.805145 31117 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.96.84:40741
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785992227050689-9601-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785992227050689-9601-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.67.keytab
--principal=kudu/127.9.96.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.96.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785992227050689-9601-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.96.67
--webserver_port=0
--tserver_master_addrs=127.9.96.126:39911
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.96.67
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785992227050689-9601-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 03:56:12 UTC on ccycloud.quasar-guijbm.root.comops.site
build id 4569618
ASAN enabled
I20260806 05:00:45.806350 31117 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 05:00:45.807760 31117 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 05:00:45.820513 31131 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 05:00:45.820574 31130 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 05:00:45.820545 31134 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 05:00:45.821553 31117 server_base.cc:1034] running on GCE node
I20260806 05:00:45.822072 31117 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 05:00:45.822916 31117 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 05:00:45.824115 31117 hybrid_clock.cc:648] HybridClock initialized: now 1785992445824044 us; error 61 us; skew 500 ppm
I20260806 05:00:45.824433 31117 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 05:00:45 dist-test-slave-x4qh krb5kdc[30534](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992445, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.96.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 05:00:45.827559 31117 init.cc:377] Logged in from keytab as kudu/127.9.96.67@KRBTEST.COM (short username kudu)
I20260806 05:00:45.828866 31117 webserver.cc:460] Webserver started at http://127.9.96.67:37687/ using document root <none> and password file <none>
I20260806 05:00:45.829490 31117 fs_manager.cc:359] Metadata directory not provided
I20260806 05:00:45.829589 31117 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 05:00:45.829823 31117 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 05:00:45.832593 31117 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785992227050689-9601-0/minicluster-data/ts-2/data/instance:
uuid: "9e35e3efb1eb4b289f33931312c99bfb"
format_stamp: "Formatted at 2026-08-06 05:00:45 on dist-test-slave-x4qh"
server_key: "12f5b45d70c62786077f37941913201a"
server_key_iv: "a9a117ada6701acebc9a8fac6fecd23e"
server_key_version: "encryptionkey@0"
I20260806 05:00:45.833297 31117 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785992227050689-9601-0/minicluster-data/ts-2/wal/instance:
uuid: "9e35e3efb1eb4b289f33931312c99bfb"
format_stamp: "Formatted at 2026-08-06 05:00:45 on dist-test-slave-x4qh"
server_key: "12f5b45d70c62786077f37941913201a"
server_key_iv: "a9a117ada6701acebc9a8fac6fecd23e"
server_key_version: "encryptionkey@0"
I20260806 05:00:45.838308 31117 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260806 05:00:45.841566 31146 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 05:00:45.842767 31117 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 05:00:45.842943 31117 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785992227050689-9601-0/minicluster-data/ts-2/data,/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785992227050689-9601-0/minicluster-data/ts-2/wal
uuid: "9e35e3efb1eb4b289f33931312c99bfb"
format_stamp: "Formatted at 2026-08-06 05:00:45 on dist-test-slave-x4qh"
server_key: "12f5b45d70c62786077f37941913201a"
server_key_iv: "a9a117ada6701acebc9a8fac6fecd23e"
server_key_version: "encryptionkey@0"
I20260806 05:00:45.843132 31117 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785992227050689-9601-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785992227050689-9601-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785992227050689-9601-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 05:00:45.861160 31117 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 05:00:45.864877 31117 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 05:00:45.865156 31117 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 05:00:45.866055 31117 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 05:00:45.867337 31117 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 05:00:45.867477 31117 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 05:00:45.867642 31117 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 05:00:45.867694 31117 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 05:00:45.879488 30906 heartbeater.cc:503] Master 127.9.96.126:39911 was elected leader, sending a full tablet report...
I20260806 05:00:45.901381 31117 rpc_server.cc:307] RPC server started. Bound to: 127.9.96.67:41983
I20260806 05:00:45.901430 31315 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.96.67:41983 every 8 connection(s)
I20260806 05:00:45.902650 31117 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785992227050689-9601-0/minicluster-data/ts-2/data/info.pb
I20260806 05:00:45.903424 9601 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu as pid 31117
I20260806 05:00:45.903631 9601 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785992227050689-9601-0/minicluster-data/ts-2/wal/instance
I20260806 05:00:45.904228 9601 external_mini_cluster.cc:1428] Setting key 38df9e775aec0dac2d551dbe33390a30
Aug 06 05:00:45 dist-test-slave-x4qh krb5kdc[30534](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992445, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.96.67@KRBTEST.COM for kudu/127.9.96.126@KRBTEST.COM
I20260806 05:00:45.922580 31318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:00:45.905292 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.96.67:46449 (local address 127.9.96.126:39911)
0806 05:00:45.905789 (+ 497us) server_negotiation.cc:206] Beginning negotiation
0806 05:00:45.905793 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 05:00:45.906900 (+ 1107us) server_negotiation.cc:407] Connection header received
0806 05:00:45.908972 (+ 2072us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 05:00:45.908978 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 05:00:45.909077 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 05:00:45.909195 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0806 05:00:45.910673 (+ 1478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:00:45.911540 (+ 867us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:00:45.913060 (+ 1520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:00:45.913458 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:00:45.913606 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:00:45.917066 (+ 3460us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 05:00:45.917071 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 05:00:45.917074 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 05:00:45.917105 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0806 05:00:45.920021 (+ 2916us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 05:00:45.920678 (+ 657us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 05:00:45.920682 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 05:00:45.920684 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 05:00:45.920796 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 05:00:45.921173 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 05:00:45.921176 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 05:00:45.921178 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 05:00:45.921420 (+ 242us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 05:00:45.921564 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0806 05:00:45.922178 (+ 614us) server_negotiation.cc:299] Negotiation successful
0806 05:00:45.922329 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":336,"thread_start_us":247,"threads_started":1}
I20260806 05:00:45.923543 31319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:00:45.905847 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:39911 (local address 127.9.96.67:46449)
0806 05:00:45.906736 (+ 889us) negotiation.cc:107] Waiting for socket to connect
0806 05:00:45.906786 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0806 05:00:45.908704 (+ 1918us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 05:00:45.909447 (+ 743us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 05:00:45.909456 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 05:00:45.909955 (+ 499us) client_negotiation.cc:189] Negotiated authn=SASL
0806 05:00:45.910482 (+ 527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 05:00:45.910517 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 05:00:45.911695 (+ 1178us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 05:00:45.911701 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 05:00:45.912903 (+ 1202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 05:00:45.912916 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 05:00:45.913706 (+ 790us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 05:00:45.913712 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 05:00:45.913915 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:00:45.914585 (+ 670us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 05:00:45.914654 (+ 69us) client_negotiation.cc:656] calling sasl_client_start()
0806 05:00:45.916882 (+ 2228us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 05:00:45.920156 (+ 3274us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 05:00:45.920163 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 05:00:45.920174 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0806 05:00:45.920576 (+ 402us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 05:00:45.920905 (+ 329us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 05:00:45.920909 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 05:00:45.920911 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 05:00:45.921079 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 05:00:45.921551 (+ 472us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 05:00:45.921559 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 05:00:45.921858 (+ 299us) client_negotiation.cc:769] Sending connection context
0806 05:00:45.922094 (+ 236us) client_negotiation.cc:240] Negotiation successful
0806 05:00:45.922436 (+ 342us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":577,"thread_start_us":319,"threads_started":1}
I20260806 05:00:45.924842 31316 heartbeater.cc:347] Connected to a master server at 127.9.96.126:39911
I20260806 05:00:45.925128 31316 heartbeater.cc:464] Registering TS with master...
I20260806 05:00:45.925834 31316 heartbeater.cc:511] Master 127.9.96.126:39911 requested a full tablet report, sending...
I20260806 05:00:45.927213 30607 ts_manager.cc:194] Registered new tserver with Master: 9e35e3efb1eb4b289f33931312c99bfb (127.9.96.67:41983)
I20260806 05:00:45.928032 30607 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.96.67@KRBTEST.COM'} at 127.9.96.67:46449
I20260806 05:00:45.932094 9601 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 05:00:45.948608 31318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:00:45.936560 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48776 (local address 127.9.96.126:39911)
0806 05:00:45.936764 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0806 05:00:45.936773 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 05:00:45.937195 (+ 422us) server_negotiation.cc:407] Connection header received
0806 05:00:45.937318 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 05:00:45.937322 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 05:00:45.937384 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 05:00:45.937460 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0806 05:00:45.938791 (+ 1331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:00:45.939587 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:00:45.940877 (+ 1290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:00:45.941253 (+ 376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:00:45.941340 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:00:45.943683 (+ 2343us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 05:00:45.943688 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 05:00:45.943691 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 05:00:45.943718 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0806 05:00:45.946012 (+ 2294us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 05:00:45.946654 (+ 642us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 05:00:45.946660 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 05:00:45.946664 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 05:00:45.946784 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 05:00:45.947188 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 05:00:45.947196 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 05:00:45.947200 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 05:00:45.947625 (+ 425us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 05:00:45.947819 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0806 05:00:45.948149 (+ 330us) server_negotiation.cc:299] Negotiation successful
0806 05:00:45.948314 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260806 05:00:45.952310 30607 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48776:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 05:00:45.959354 30607 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 05:00:45.988725 31340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:00:45.973016 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54292 (local address 127.9.96.65:38017)
0806 05:00:45.977197 (+ 4181us) server_negotiation.cc:206] Beginning negotiation
0806 05:00:45.977210 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 05:00:45.977249 (+ 39us) server_negotiation.cc:407] Connection header received
0806 05:00:45.977659 (+ 410us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 05:00:45.977671 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 05:00:45.978001 (+ 330us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 05:00:45.978238 (+ 237us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 05:00:45.979489 (+ 1251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:00:45.982948 (+ 3459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:00:45.985960 (+ 3012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:00:45.987186 (+ 1226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:00:45.987348 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:00:45.987635 (+ 287us) server_negotiation.cc:1035] Waiting for connection context
0806 05:00:45.988219 (+ 584us) server_negotiation.cc:299] Negotiation successful
0806 05:00:45.988395 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2799,"thread_start_us":2599,"threads_started":1}
I20260806 05:00:45.989056 31339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:00:45.972174 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.65:38017 (local address 127.0.0.1:54292)
0806 05:00:45.976394 (+ 4220us) negotiation.cc:107] Waiting for socket to connect
0806 05:00:45.976418 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0806 05:00:45.977411 (+ 993us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 05:00:45.978625 (+ 1214us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 05:00:45.978636 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 05:00:45.978678 (+ 42us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 05:00:45.979191 (+ 513us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 05:00:45.979207 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 05:00:45.983145 (+ 3938us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 05:00:45.983154 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 05:00:45.985795 (+ 2641us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 05:00:45.985808 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 05:00:45.987528 (+ 1720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 05:00:45.987536 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 05:00:45.987940 (+ 404us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:00:45.988010 (+ 70us) client_negotiation.cc:769] Sending connection context
0806 05:00:45.988150 (+ 140us) client_negotiation.cc:240] Negotiation successful
0806 05:00:45.988336 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4042,"mutex_wait_us":100,"thread_start_us":3818,"threads_started":1}
I20260806 05:00:45.991731 31336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:00:45.972034 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45128 (local address 127.9.96.66:43613)
0806 05:00:45.983457 (+ 11423us) server_negotiation.cc:206] Beginning negotiation
0806 05:00:45.983465 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 05:00:45.983495 (+ 30us) server_negotiation.cc:407] Connection header received
0806 05:00:45.983598 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 05:00:45.983605 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 05:00:45.983823 (+ 218us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 05:00:45.984031 (+ 208us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 05:00:45.985081 (+ 1050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:00:45.986586 (+ 1505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:00:45.989362 (+ 2776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:00:45.990658 (+ 1296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:00:45.990809 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:00:45.990999 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0806 05:00:45.991227 (+ 228us) server_negotiation.cc:299] Negotiation successful
0806 05:00:45.991435 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11224,"thread_start_us":11116,"threads_started":1}
I20260806 05:00:45.992237 31341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:00:45.976601 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.67:41983 (local address 127.0.0.1:33196)
0806 05:00:45.983176 (+ 6575us) negotiation.cc:107] Waiting for socket to connect
0806 05:00:45.983196 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0806 05:00:45.983395 (+ 199us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 05:00:45.983924 (+ 529us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 05:00:45.983928 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 05:00:45.983954 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 05:00:45.984520 (+ 566us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 05:00:45.984534 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 05:00:45.986698 (+ 2164us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 05:00:45.986704 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 05:00:45.989116 (+ 2412us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 05:00:45.989132 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 05:00:45.991220 (+ 2088us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 05:00:45.991226 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 05:00:45.991458 (+ 232us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:00:45.991500 (+ 42us) client_negotiation.cc:769] Sending connection context
0806 05:00:45.991595 (+ 95us) client_negotiation.cc:240] Negotiation successful
0806 05:00:45.991719 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6424,"mutex_wait_us":159,"thread_start_us":6236,"threads_started":1}
I20260806 05:00:45.992309 31343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:00:45.976730 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33196 (local address 127.9.96.67:41983)
0806 05:00:45.982832 (+ 6102us) server_negotiation.cc:206] Beginning negotiation
0806 05:00:45.982843 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 05:00:45.983420 (+ 577us) server_negotiation.cc:407] Connection header received
0806 05:00:45.983542 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 05:00:45.983550 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 05:00:45.983732 (+ 182us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 05:00:45.983942 (+ 210us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 05:00:45.984770 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:00:45.986541 (+ 1771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:00:45.989300 (+ 2759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:00:45.990726 (+ 1426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:00:45.990851 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:00:45.991067 (+ 216us) server_negotiation.cc:1035] Waiting for connection context
0806 05:00:45.991857 (+ 790us) server_negotiation.cc:299] Negotiation successful
0806 05:00:45.992016 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5915,"thread_start_us":5769,"threads_started":1}
I20260806 05:00:45.991763 31335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:00:45.971717 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.66:43613 (local address 127.0.0.1:45128)
0806 05:00:45.972869 (+ 1152us) negotiation.cc:107] Waiting for socket to connect
0806 05:00:45.972913 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0806 05:00:45.973068 (+ 155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 05:00:45.984241 (+ 11173us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 05:00:45.984247 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 05:00:45.984278 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 05:00:45.984859 (+ 581us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 05:00:45.984878 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 05:00:45.986766 (+ 1888us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 05:00:45.986772 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 05:00:45.989145 (+ 2373us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 05:00:45.989164 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 05:00:45.990866 (+ 1702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 05:00:45.990872 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 05:00:45.991032 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:00:45.991069 (+ 37us) client_negotiation.cc:769] Sending connection context
0806 05:00:45.991184 (+ 115us) client_negotiation.cc:240] Negotiation successful
0806 05:00:45.991313 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":932,"mutex_wait_us":21,"thread_start_us":805,"threads_started":1}
I20260806 05:00:45.993512 30807 tablet_service.cc:1467] Processing CreateTablet for tablet 248caed6579e49e8992e2265dd991800 (DEFAULT_TABLE table=test-table [id=a622a37cd1244fe3be96dddf720d456d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 05:00:45.995025 30807 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 248caed6579e49e8992e2265dd991800. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 05:00:45.995949 31212 tablet_service.cc:1467] Processing CreateTablet for tablet 248caed6579e49e8992e2265dd991800 (DEFAULT_TABLE table=test-table [id=a622a37cd1244fe3be96dddf720d456d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 05:00:45.996201 31010 tablet_service.cc:1467] Processing CreateTablet for tablet 248caed6579e49e8992e2265dd991800 (DEFAULT_TABLE table=test-table [id=a622a37cd1244fe3be96dddf720d456d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 05:00:45.997339 31212 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 248caed6579e49e8992e2265dd991800. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 05:00:45.997404 31010 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 248caed6579e49e8992e2265dd991800. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 05:00:46.006373 31346 tablet_bootstrap.cc:492] T 248caed6579e49e8992e2265dd991800 P 7bf95bbc6b8240e4b043b56c9cf673dc: Bootstrap starting.
I20260806 05:00:46.009485 31346 tablet_bootstrap.cc:654] T 248caed6579e49e8992e2265dd991800 P 7bf95bbc6b8240e4b043b56c9cf673dc: Neither blocks nor log segments found. Creating new log.
I20260806 05:00:46.010608 31346 log.cc:824] T 248caed6579e49e8992e2265dd991800 P 7bf95bbc6b8240e4b043b56c9cf673dc: Log is configured to *not* fsync() on all Append() calls
I20260806 05:00:46.013063 31351 tablet_bootstrap.cc:492] T 248caed6579e49e8992e2265dd991800 P 2f286e3a16b847239e12defeff122bec: Bootstrap starting.
I20260806 05:00:46.017700 31351 tablet_bootstrap.cc:654] T 248caed6579e49e8992e2265dd991800 P 2f286e3a16b847239e12defeff122bec: Neither blocks nor log segments found. Creating new log.
I20260806 05:00:46.018904 31350 tablet_bootstrap.cc:492] T 248caed6579e49e8992e2265dd991800 P 9e35e3efb1eb4b289f33931312c99bfb: Bootstrap starting.
I20260806 05:00:46.019066 31351 log.cc:824] T 248caed6579e49e8992e2265dd991800 P 2f286e3a16b847239e12defeff122bec: Log is configured to *not* fsync() on all Append() calls
I20260806 05:00:46.020612 31346 tablet_bootstrap.cc:492] T 248caed6579e49e8992e2265dd991800 P 7bf95bbc6b8240e4b043b56c9cf673dc: No bootstrap required, opened a new log
I20260806 05:00:46.020933 31346 ts_tablet_manager.cc:1397] T 248caed6579e49e8992e2265dd991800 P 7bf95bbc6b8240e4b043b56c9cf673dc: Time spent bootstrapping tablet: real 0.015s user 0.008s sys 0.005s
I20260806 05:00:46.022569 31350 tablet_bootstrap.cc:654] T 248caed6579e49e8992e2265dd991800 P 9e35e3efb1eb4b289f33931312c99bfb: Neither blocks nor log segments found. Creating new log.
I20260806 05:00:46.023294 31351 tablet_bootstrap.cc:492] T 248caed6579e49e8992e2265dd991800 P 2f286e3a16b847239e12defeff122bec: No bootstrap required, opened a new log
I20260806 05:00:46.023586 31351 ts_tablet_manager.cc:1397] T 248caed6579e49e8992e2265dd991800 P 2f286e3a16b847239e12defeff122bec: Time spent bootstrapping tablet: real 0.011s user 0.010s sys 0.000s
I20260806 05:00:46.023640 31350 log.cc:824] T 248caed6579e49e8992e2265dd991800 P 9e35e3efb1eb4b289f33931312c99bfb: Log is configured to *not* fsync() on all Append() calls
I20260806 05:00:46.026023 31350 tablet_bootstrap.cc:492] T 248caed6579e49e8992e2265dd991800 P 9e35e3efb1eb4b289f33931312c99bfb: No bootstrap required, opened a new log
I20260806 05:00:46.026273 31350 ts_tablet_manager.cc:1397] T 248caed6579e49e8992e2265dd991800 P 9e35e3efb1eb4b289f33931312c99bfb: Time spent bootstrapping tablet: real 0.008s user 0.003s sys 0.003s
I20260806 05:00:46.031782 31346 raft_consensus.cc:348] T 248caed6579e49e8992e2265dd991800 P 7bf95bbc6b8240e4b043b56c9cf673dc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f286e3a16b847239e12defeff122bec" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 43613 } } peers { permanent_uuid: "7bf95bbc6b8240e4b043b56c9cf673dc" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 38017 } } peers { permanent_uuid: "9e35e3efb1eb4b289f33931312c99bfb" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 41983 } }
I20260806 05:00:46.032215 31346 raft_consensus.cc:374] T 248caed6579e49e8992e2265dd991800 P 7bf95bbc6b8240e4b043b56c9cf673dc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 05:00:46.032326 31346 raft_consensus.cc:729] T 248caed6579e49e8992e2265dd991800 P 7bf95bbc6b8240e4b043b56c9cf673dc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7bf95bbc6b8240e4b043b56c9cf673dc, State: Initialized, Role: FOLLOWER
I20260806 05:00:46.032961 31346 consensus_queue.cc:260] T 248caed6579e49e8992e2265dd991800 P 7bf95bbc6b8240e4b043b56c9cf673dc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f286e3a16b847239e12defeff122bec" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 43613 } } peers { permanent_uuid: "7bf95bbc6b8240e4b043b56c9cf673dc" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 38017 } } peers { permanent_uuid: "9e35e3efb1eb4b289f33931312c99bfb" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 41983 } }
I20260806 05:00:46.033439 31350 raft_consensus.cc:348] T 248caed6579e49e8992e2265dd991800 P 9e35e3efb1eb4b289f33931312c99bfb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f286e3a16b847239e12defeff122bec" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 43613 } } peers { permanent_uuid: "7bf95bbc6b8240e4b043b56c9cf673dc" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 38017 } } peers { permanent_uuid: "9e35e3efb1eb4b289f33931312c99bfb" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 41983 } }
I20260806 05:00:46.033923 31350 raft_consensus.cc:374] T 248caed6579e49e8992e2265dd991800 P 9e35e3efb1eb4b289f33931312c99bfb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 05:00:46.034032 31350 raft_consensus.cc:729] T 248caed6579e49e8992e2265dd991800 P 9e35e3efb1eb4b289f33931312c99bfb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e35e3efb1eb4b289f33931312c99bfb, State: Initialized, Role: FOLLOWER
I20260806 05:00:46.034009 31351 raft_consensus.cc:348] T 248caed6579e49e8992e2265dd991800 P 2f286e3a16b847239e12defeff122bec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f286e3a16b847239e12defeff122bec" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 43613 } } peers { permanent_uuid: "7bf95bbc6b8240e4b043b56c9cf673dc" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 38017 } } peers { permanent_uuid: "9e35e3efb1eb4b289f33931312c99bfb" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 41983 } }
I20260806 05:00:46.034418 31351 raft_consensus.cc:374] T 248caed6579e49e8992e2265dd991800 P 2f286e3a16b847239e12defeff122bec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 05:00:46.034520 31351 raft_consensus.cc:729] T 248caed6579e49e8992e2265dd991800 P 2f286e3a16b847239e12defeff122bec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2f286e3a16b847239e12defeff122bec, State: Initialized, Role: FOLLOWER
I20260806 05:00:46.034924 31346 ts_tablet_manager.cc:1428] T 248caed6579e49e8992e2265dd991800 P 7bf95bbc6b8240e4b043b56c9cf673dc: Time spent starting tablet: real 0.014s user 0.008s sys 0.007s
I20260806 05:00:46.034737 31350 consensus_queue.cc:260] T 248caed6579e49e8992e2265dd991800 P 9e35e3efb1eb4b289f33931312c99bfb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f286e3a16b847239e12defeff122bec" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 43613 } } peers { permanent_uuid: "7bf95bbc6b8240e4b043b56c9cf673dc" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 38017 } } peers { permanent_uuid: "9e35e3efb1eb4b289f33931312c99bfb" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 41983 } }
I20260806 05:00:46.035080 31351 consensus_queue.cc:260] T 248caed6579e49e8992e2265dd991800 P 2f286e3a16b847239e12defeff122bec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f286e3a16b847239e12defeff122bec" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 43613 } } peers { permanent_uuid: "7bf95bbc6b8240e4b043b56c9cf673dc" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 38017 } } peers { permanent_uuid: "9e35e3efb1eb4b289f33931312c99bfb" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 41983 } }
I20260806 05:00:46.036528 31316 heartbeater.cc:503] Master 127.9.96.126:39911 was elected leader, sending a full tablet report...
I20260806 05:00:46.037071 31350 ts_tablet_manager.cc:1428] T 248caed6579e49e8992e2265dd991800 P 9e35e3efb1eb4b289f33931312c99bfb: Time spent starting tablet: real 0.011s user 0.010s sys 0.001s
I20260806 05:00:46.038125 31108 heartbeater.cc:503] Master 127.9.96.126:39911 was elected leader, sending a full tablet report...
I20260806 05:00:46.038807 31351 ts_tablet_manager.cc:1428] T 248caed6579e49e8992e2265dd991800 P 2f286e3a16b847239e12defeff122bec: Time spent starting tablet: real 0.015s user 0.011s sys 0.004s
I20260806 05:00:46.093187 31355 raft_consensus.cc:482] T 248caed6579e49e8992e2265dd991800 P 9e35e3efb1eb4b289f33931312c99bfb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 05:00:46.093559 31355 raft_consensus.cc:504] T 248caed6579e49e8992e2265dd991800 P 9e35e3efb1eb4b289f33931312c99bfb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f286e3a16b847239e12defeff122bec" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 43613 } } peers { permanent_uuid: "7bf95bbc6b8240e4b043b56c9cf673dc" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 38017 } } peers { permanent_uuid: "9e35e3efb1eb4b289f33931312c99bfb" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 41983 } }
I20260806 05:00:46.095180 31355 leader_election.cc:288] T 248caed6579e49e8992e2265dd991800 P 9e35e3efb1eb4b289f33931312c99bfb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2f286e3a16b847239e12defeff122bec (127.9.96.66:43613), 7bf95bbc6b8240e4b043b56c9cf673dc (127.9.96.65:38017)
W20260806 05:00:46.099714 30907 tablet.cc:2369] T 248caed6579e49e8992e2265dd991800 P 7bf95bbc6b8240e4b043b56c9cf673dc: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 05:00:46.102119 31340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:00:46.095630 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.96.67:34299 (local address 127.9.96.65:38017)
0806 05:00:46.095898 (+ 268us) server_negotiation.cc:206] Beginning negotiation
0806 05:00:46.095904 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 05:00:46.096079 (+ 175us) server_negotiation.cc:407] Connection header received
0806 05:00:46.096244 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 05:00:46.096247 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 05:00:46.096308 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 05:00:46.096433 (+ 125us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 05:00:46.096864 (+ 431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:00:46.097959 (+ 1095us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:00:46.100072 (+ 2113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:00:46.101268 (+ 1196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:00:46.101423 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:00:46.101541 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0806 05:00:46.101860 (+ 319us) server_negotiation.cc:299] Negotiation successful
0806 05:00:46.101949 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":101}
I20260806 05:00:46.102216 31319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:00:46.095770 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.65:38017 (local address 127.9.96.67:34299)
0806 05:00:46.095946 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0806 05:00:46.095972 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0806 05:00:46.096122 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 05:00:46.096457 (+ 335us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 05:00:46.096462 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 05:00:46.096491 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 05:00:46.096698 (+ 207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 05:00:46.096709 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 05:00:46.098088 (+ 1379us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 05:00:46.098098 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 05:00:46.099903 (+ 1805us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 05:00:46.099921 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 05:00:46.101454 (+ 1533us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 05:00:46.101461 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 05:00:46.101673 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:00:46.101712 (+ 39us) client_negotiation.cc:769] Sending connection context
0806 05:00:46.101808 (+ 96us) client_negotiation.cc:240] Negotiation successful
0806 05:00:46.101939 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34,"mutex_wait_us":1}
I20260806 05:00:46.103250 30837 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "248caed6579e49e8992e2265dd991800" candidate_uuid: "9e35e3efb1eb4b289f33931312c99bfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7bf95bbc6b8240e4b043b56c9cf673dc" is_pre_election: true
I20260806 05:00:46.103722 31336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:00:46.095792 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.96.67:38431 (local address 127.9.96.66:43613)
0806 05:00:46.095979 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0806 05:00:46.095983 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 05:00:46.097126 (+ 1143us) server_negotiation.cc:407] Connection header received
0806 05:00:46.097341 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 05:00:46.097362 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 05:00:46.097433 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 05:00:46.097518 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 05:00:46.098499 (+ 981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:00:46.099275 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:00:46.101503 (+ 2228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:00:46.102714 (+ 1211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:00:46.102858 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:00:46.102924 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0806 05:00:46.103320 (+ 396us) server_negotiation.cc:299] Negotiation successful
0806 05:00:46.103482 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260806 05:00:46.103825 31357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:00:46.095770 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.66:43613 (local address 127.9.96.67:38431)
0806 05:00:46.096989 (+ 1219us) negotiation.cc:107] Waiting for socket to connect
0806 05:00:46.097008 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0806 05:00:46.097173 (+ 165us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 05:00:46.097672 (+ 499us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 05:00:46.097677 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 05:00:46.097701 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 05:00:46.098320 (+ 619us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 05:00:46.098340 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 05:00:46.099441 (+ 1101us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 05:00:46.099447 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 05:00:46.101342 (+ 1895us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 05:00:46.101359 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 05:00:46.102938 (+ 1579us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 05:00:46.102944 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 05:00:46.103150 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:00:46.103180 (+ 30us) client_negotiation.cc:769] Sending connection context
0806 05:00:46.103280 (+ 100us) client_negotiation.cc:240] Negotiation successful
0806 05:00:46.103438 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1078,"mutex_wait_us":430,"thread_start_us":446,"threads_started":1}
I20260806 05:00:46.103868 30837 raft_consensus.cc:2454] T 248caed6579e49e8992e2265dd991800 P 7bf95bbc6b8240e4b043b56c9cf673dc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e35e3efb1eb4b289f33931312c99bfb in term 0.
I20260806 05:00:46.104597 31148 leader_election.cc:302] T 248caed6579e49e8992e2265dd991800 P 9e35e3efb1eb4b289f33931312c99bfb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bf95bbc6b8240e4b043b56c9cf673dc, 9e35e3efb1eb4b289f33931312c99bfb; no voters:
I20260806 05:00:46.104985 31355 raft_consensus.cc:2781] T 248caed6579e49e8992e2265dd991800 P 9e35e3efb1eb4b289f33931312c99bfb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 05:00:46.105113 31355 raft_consensus.cc:482] T 248caed6579e49e8992e2265dd991800 P 9e35e3efb1eb4b289f33931312c99bfb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 05:00:46.104974 31039 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "248caed6579e49e8992e2265dd991800" candidate_uuid: "9e35e3efb1eb4b289f33931312c99bfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2f286e3a16b847239e12defeff122bec" is_pre_election: true
I20260806 05:00:46.105248 31355 raft_consensus.cc:3037] T 248caed6579e49e8992e2265dd991800 P 9e35e3efb1eb4b289f33931312c99bfb [term 0 FOLLOWER]: Advancing to term 1
I20260806 05:00:46.105484 31039 raft_consensus.cc:2454] T 248caed6579e49e8992e2265dd991800 P 2f286e3a16b847239e12defeff122bec [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e35e3efb1eb4b289f33931312c99bfb in term 0.
I20260806 05:00:46.107854 31355 raft_consensus.cc:504] T 248caed6579e49e8992e2265dd991800 P 9e35e3efb1eb4b289f33931312c99bfb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f286e3a16b847239e12defeff122bec" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 43613 } } peers { permanent_uuid: "7bf95bbc6b8240e4b043b56c9cf673dc" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 38017 } } peers { permanent_uuid: "9e35e3efb1eb4b289f33931312c99bfb" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 41983 } }
I20260806 05:00:46.108507 31355 leader_election.cc:288] T 248caed6579e49e8992e2265dd991800 P 9e35e3efb1eb4b289f33931312c99bfb [CANDIDATE]: Term 1 election: Requested vote from peers 2f286e3a16b847239e12defeff122bec (127.9.96.66:43613), 7bf95bbc6b8240e4b043b56c9cf673dc (127.9.96.65:38017)
I20260806 05:00:46.108853 31039 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "248caed6579e49e8992e2265dd991800" candidate_uuid: "9e35e3efb1eb4b289f33931312c99bfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2f286e3a16b847239e12defeff122bec"
I20260806 05:00:46.109043 31039 raft_consensus.cc:3037] T 248caed6579e49e8992e2265dd991800 P 2f286e3a16b847239e12defeff122bec [term 0 FOLLOWER]: Advancing to term 1
I20260806 05:00:46.109041 30837 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "248caed6579e49e8992e2265dd991800" candidate_uuid: "9e35e3efb1eb4b289f33931312c99bfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7bf95bbc6b8240e4b043b56c9cf673dc"
I20260806 05:00:46.109236 30837 raft_consensus.cc:3037] T 248caed6579e49e8992e2265dd991800 P 7bf95bbc6b8240e4b043b56c9cf673dc [term 0 FOLLOWER]: Advancing to term 1
I20260806 05:00:46.111289 31039 raft_consensus.cc:2454] T 248caed6579e49e8992e2265dd991800 P 2f286e3a16b847239e12defeff122bec [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e35e3efb1eb4b289f33931312c99bfb in term 1.
I20260806 05:00:46.111441 30837 raft_consensus.cc:2454] T 248caed6579e49e8992e2265dd991800 P 7bf95bbc6b8240e4b043b56c9cf673dc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e35e3efb1eb4b289f33931312c99bfb in term 1.
I20260806 05:00:46.111714 31151 leader_election.cc:302] T 248caed6579e49e8992e2265dd991800 P 9e35e3efb1eb4b289f33931312c99bfb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2f286e3a16b847239e12defeff122bec, 9e35e3efb1eb4b289f33931312c99bfb; no voters:
I20260806 05:00:46.112028 31355 raft_consensus.cc:2781] T 248caed6579e49e8992e2265dd991800 P 9e35e3efb1eb4b289f33931312c99bfb [term 1 FOLLOWER]: Leader election won for term 1
I20260806 05:00:46.112466 31355 raft_consensus.cc:686] T 248caed6579e49e8992e2265dd991800 P 9e35e3efb1eb4b289f33931312c99bfb [term 1 LEADER]: Becoming Leader. State: Replica: 9e35e3efb1eb4b289f33931312c99bfb, State: Running, Role: LEADER
I20260806 05:00:46.113018 31355 consensus_queue.cc:237] T 248caed6579e49e8992e2265dd991800 P 9e35e3efb1eb4b289f33931312c99bfb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f286e3a16b847239e12defeff122bec" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 43613 } } peers { permanent_uuid: "7bf95bbc6b8240e4b043b56c9cf673dc" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 38017 } } peers { permanent_uuid: "9e35e3efb1eb4b289f33931312c99bfb" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 41983 } }
I20260806 05:00:46.118016 30603 catalog_manager.cc:5495] T 248caed6579e49e8992e2265dd991800 P 9e35e3efb1eb4b289f33931312c99bfb reported cstate change: term changed from 0 to 1, leader changed from <none> to 9e35e3efb1eb4b289f33931312c99bfb (127.9.96.67). New cstate: current_term: 1 leader_uuid: "9e35e3efb1eb4b289f33931312c99bfb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f286e3a16b847239e12defeff122bec" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 43613 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7bf95bbc6b8240e4b043b56c9cf673dc" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 38017 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e35e3efb1eb4b289f33931312c99bfb" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 41983 } health_report { overall_health: HEALTHY } } }
W20260806 05:00:46.128252 31109 tablet.cc:2369] T 248caed6579e49e8992e2265dd991800 P 2f286e3a16b847239e12defeff122bec: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 05:00:46.140547 31343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:00:46.135028 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33208 (local address 127.9.96.67:41983)
0806 05:00:46.135237 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0806 05:00:46.135243 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 05:00:46.135350 (+ 107us) server_negotiation.cc:407] Connection header received
0806 05:00:46.135612 (+ 262us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 05:00:46.135616 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 05:00:46.135740 (+ 124us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 05:00:46.135924 (+ 184us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 05:00:46.136292 (+ 368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:00:46.137003 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:00:46.138515 (+ 1512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:00:46.138926 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:00:46.139059 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:00:46.139564 (+ 505us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 05:00:46.140000 (+ 436us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 05:00:46.140161 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0806 05:00:46.140222 (+ 61us) server_negotiation.cc:299] Negotiation successful
0806 05:00:46.140298 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260806 05:00:46.148130 31039 raft_consensus.cc:1261] T 248caed6579e49e8992e2265dd991800 P 2f286e3a16b847239e12defeff122bec [term 1 FOLLOWER]: Refusing update from remote peer 9e35e3efb1eb4b289f33931312c99bfb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 05:00:46.148289 30837 raft_consensus.cc:1261] T 248caed6579e49e8992e2265dd991800 P 7bf95bbc6b8240e4b043b56c9cf673dc [term 1 FOLLOWER]: Refusing update from remote peer 9e35e3efb1eb4b289f33931312c99bfb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 05:00:46.149197 31355 consensus_queue.cc:1035] T 248caed6579e49e8992e2265dd991800 P 9e35e3efb1eb4b289f33931312c99bfb [LEADER]: Connected to new peer: Peer: permanent_uuid: "2f286e3a16b847239e12defeff122bec" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 43613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 05:00:46.149500 31358 consensus_queue.cc:1035] T 248caed6579e49e8992e2265dd991800 P 9e35e3efb1eb4b289f33931312c99bfb [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bf95bbc6b8240e4b043b56c9cf673dc" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 38017 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260806 05:00:46.154524 31317 tablet.cc:2369] T 248caed6579e49e8992e2265dd991800 P 9e35e3efb1eb4b289f33931312c99bfb: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 05:00:46.160091 31366 mvcc.cc:205] Tried to move back new op lower bound from 7315425059412844544 to 7315425059283111936. Current Snapshot: MvccSnapshot[applied={T|T < 7315425059412844544}]
I20260806 05:00:46.177428 30603 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48776:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 05:00:46.177695 30603 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48776:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 05:00:46.181267 30603 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 245b5b6f6eca4e65b1c4e9298645efec: Sending DeleteTablet for 3 replicas of tablet 248caed6579e49e8992e2265dd991800
I20260806 05:00:46.182319 31010 tablet_service.cc:1514] Processing DeleteTablet for tablet 248caed6579e49e8992e2265dd991800 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 05:00:46 UTC) from {username='kudu', principal='kudu/127.9.96.126@KRBTEST.COM'} at 127.0.0.1:45128
I20260806 05:00:46.182451 31212 tablet_service.cc:1514] Processing DeleteTablet for tablet 248caed6579e49e8992e2265dd991800 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 05:00:46 UTC) from {username='kudu', principal='kudu/127.9.96.126@KRBTEST.COM'} at 127.0.0.1:33196
I20260806 05:00:46.182508 30807 tablet_service.cc:1514] Processing DeleteTablet for tablet 248caed6579e49e8992e2265dd991800 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 05:00:46 UTC) from {username='kudu', principal='kudu/127.9.96.126@KRBTEST.COM'} at 127.0.0.1:54292
I20260806 05:00:46.183741 9601 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu pid 30707
I20260806 05:00:46.198140 31388 tablet_replica.cc:330] T 248caed6579e49e8992e2265dd991800 P 9e35e3efb1eb4b289f33931312c99bfb: stopping tablet replica
I20260806 05:00:46.198693 31388 raft_consensus.cc:2229] T 248caed6579e49e8992e2265dd991800 P 9e35e3efb1eb4b289f33931312c99bfb [term 1 LEADER]: Raft consensus shutting down.
I20260806 05:00:46.199446 31388 raft_consensus.cc:2258] T 248caed6579e49e8992e2265dd991800 P 9e35e3efb1eb4b289f33931312c99bfb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 05:00:46.200295 31391 tablet_replica.cc:330] T 248caed6579e49e8992e2265dd991800 P 7bf95bbc6b8240e4b043b56c9cf673dc: stopping tablet replica
I20260806 05:00:46.200722 31391 raft_consensus.cc:2229] T 248caed6579e49e8992e2265dd991800 P 7bf95bbc6b8240e4b043b56c9cf673dc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 05:00:46.201273 31391 raft_consensus.cc:2258] T 248caed6579e49e8992e2265dd991800 P 7bf95bbc6b8240e4b043b56c9cf673dc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 05:00:46.201315 31387 tablet_replica.cc:330] T 248caed6579e49e8992e2265dd991800 P 2f286e3a16b847239e12defeff122bec: stopping tablet replica
I20260806 05:00:46.201682 31387 raft_consensus.cc:2229] T 248caed6579e49e8992e2265dd991800 P 2f286e3a16b847239e12defeff122bec [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 05:00:46.202126 31387 raft_consensus.cc:2258] T 248caed6579e49e8992e2265dd991800 P 2f286e3a16b847239e12defeff122bec [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 05:00:46.204555 31388 ts_tablet_manager.cc:1905] T 248caed6579e49e8992e2265dd991800 P 9e35e3efb1eb4b289f33931312c99bfb: Deleting tablet data with delete state TABLET_DATA_DELETED
Aug 06 05:00:46 dist-test-slave-x4qh krb5kdc[30534](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992443, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.96.65@KRBTEST.COM
I20260806 05:00:46.212102 31388 ts_tablet_manager.cc:1918] T 248caed6579e49e8992e2265dd991800 P 9e35e3efb1eb4b289f33931312c99bfb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 05:00:46.212289 31388 log.cc:1196] T 248caed6579e49e8992e2265dd991800 P 9e35e3efb1eb4b289f33931312c99bfb: Deleting WAL directory at /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785992227050689-9601-0/minicluster-data/ts-2/wal/wals/248caed6579e49e8992e2265dd991800
I20260806 05:00:46.212872 31388 ts_tablet_manager.cc:1939] T 248caed6579e49e8992e2265dd991800 P 9e35e3efb1eb4b289f33931312c99bfb: Deleting consensus metadata
I20260806 05:00:46.214707 30581 catalog_manager.cc:4841] TS 9e35e3efb1eb4b289f33931312c99bfb (127.9.96.67:41983): tablet 248caed6579e49e8992e2265dd991800 (table test-table [id=a622a37cd1244fe3be96dddf720d456d]) successfully deleted
I20260806 05:00:46.215656 31387 ts_tablet_manager.cc:1905] T 248caed6579e49e8992e2265dd991800 P 2f286e3a16b847239e12defeff122bec: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 05:00:46.216140 31391 ts_tablet_manager.cc:1905] T 248caed6579e49e8992e2265dd991800 P 7bf95bbc6b8240e4b043b56c9cf673dc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 05:00:46.221675 31340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:00:46.185151 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54308 (local address 127.9.96.65:38017)
0806 05:00:46.185362 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0806 05:00:46.185371 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 05:00:46.200662 (+ 15291us) server_negotiation.cc:407] Connection header received
0806 05:00:46.200886 (+ 224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 05:00:46.200892 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 05:00:46.200992 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 05:00:46.201096 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0806 05:00:46.203612 (+ 2516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:00:46.204724 (+ 1112us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:00:46.206764 (+ 2040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:00:46.207407 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:00:46.207497 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:00:46.213370 (+ 5873us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 05:00:46.213385 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 05:00:46.213406 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 05:00:46.213451 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0806 05:00:46.218173 (+ 4722us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 05:00:46.219129 (+ 956us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 05:00:46.219138 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 05:00:46.219142 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 05:00:46.219299 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 05:00:46.219776 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 05:00:46.219782 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 05:00:46.219785 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 05:00:46.220216 (+ 431us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 05:00:46.220492 (+ 276us) server_negotiation.cc:1035] Waiting for connection context
0806 05:00:46.221079 (+ 587us) server_negotiation.cc:299] Negotiation successful
0806 05:00:46.221301 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260806 05:00:46.222448 31387 ts_tablet_manager.cc:1918] T 248caed6579e49e8992e2265dd991800 P 2f286e3a16b847239e12defeff122bec: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 05:00:46.222608 31387 log.cc:1196] T 248caed6579e49e8992e2265dd991800 P 2f286e3a16b847239e12defeff122bec: Deleting WAL directory at /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785992227050689-9601-0/minicluster-data/ts-1/wal/wals/248caed6579e49e8992e2265dd991800
I20260806 05:00:46.223152 31387 ts_tablet_manager.cc:1939] T 248caed6579e49e8992e2265dd991800 P 2f286e3a16b847239e12defeff122bec: Deleting consensus metadata
I20260806 05:00:46.223345 30896 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 05:00:46.224977 30583 catalog_manager.cc:4841] TS 2f286e3a16b847239e12defeff122bec (127.9.96.66:43613): tablet 248caed6579e49e8992e2265dd991800 (table test-table [id=a622a37cd1244fe3be96dddf720d456d]) successfully deleted
I20260806 05:00:46.328959 9601 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu with pid 30707
W20260806 05:00:46.355708 30581 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.9.96.65:38017 (error 108)
I20260806 05:00:46.356602 9601 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu pid 30916
I20260806 05:00:46.357966 31335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:00:46.357532 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.65:38017 (local address 127.0.0.1:54316)
0806 05:00:46.357728 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0806 05:00:46.357830 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.96.65:38017: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
W20260806 05:00:46.358330 30581 catalog_manager.cc:4584] TS 7bf95bbc6b8240e4b043b56c9cf673dc (127.9.96.65:38017): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 248caed6579e49e8992e2265dd991800: Network error: Client connection negotiation failed: client connection to 127.9.96.65:38017: connect: Connection refused (error 111)
Aug 06 05:00:46 dist-test-slave-x4qh krb5kdc[30534](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992443, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.96.66@KRBTEST.COM
I20260806 05:00:46.378696 31336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:00:46.357510 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45136 (local address 127.9.96.66:43613)
0806 05:00:46.358263 (+ 753us) server_negotiation.cc:206] Beginning negotiation
0806 05:00:46.358267 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 05:00:46.358284 (+ 17us) server_negotiation.cc:407] Connection header received
0806 05:00:46.358338 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 05:00:46.358342 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 05:00:46.358422 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 05:00:46.358551 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0806 05:00:46.360055 (+ 1504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:00:46.360972 (+ 917us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:00:46.362249 (+ 1277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:00:46.362660 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:00:46.362802 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:00:46.371444 (+ 8642us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 05:00:46.371462 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 05:00:46.371481 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 05:00:46.371522 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0806 05:00:46.375222 (+ 3700us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 05:00:46.376138 (+ 916us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 05:00:46.376149 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 05:00:46.376156 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 05:00:46.376308 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 05:00:46.376762 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 05:00:46.376767 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 05:00:46.376770 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 05:00:46.377232 (+ 462us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 05:00:46.377550 (+ 318us) server_negotiation.cc:1035] Waiting for connection context
0806 05:00:46.377915 (+ 365us) server_negotiation.cc:299] Negotiation successful
0806 05:00:46.378153 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":627}
I20260806 05:00:46.379455 31100 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 05:00:46.412349 31335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:00:46.411784 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.65:38017 (local address 127.0.0.1:54318)
0806 05:00:46.412039 (+ 255us) negotiation.cc:107] Waiting for socket to connect
0806 05:00:46.412157 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.96.65:38017: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101}
I20260806 05:00:46.413617 31335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:00:46.413176 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.65:38017 (local address 127.0.0.1:54330)
0806 05:00:46.413357 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0806 05:00:46.413464 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.96.65:38017: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260806 05:00:46.471057 9601 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu with pid 30916
I20260806 05:00:46.474503 31335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:00:46.473938 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.65:38017 (local address 127.0.0.1:54334)
0806 05:00:46.474189 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0806 05:00:46.474302 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.96.65:38017: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":88}
I20260806 05:00:46.475816 31335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:00:46.475411 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.65:38017 (local address 127.0.0.1:54346)
0806 05:00:46.475586 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0806 05:00:46.475665 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.96.65:38017: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260806 05:00:46.501529 9601 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu pid 31117
Aug 06 05:00:46 dist-test-slave-x4qh krb5kdc[30534](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992443, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.96.67@KRBTEST.COM
I20260806 05:00:46.518239 31343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:00:46.502373 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33212 (local address 127.9.96.67:41983)
0806 05:00:46.502546 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0806 05:00:46.502551 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 05:00:46.502630 (+ 79us) server_negotiation.cc:407] Connection header received
0806 05:00:46.502837 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 05:00:46.502841 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 05:00:46.502919 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 05:00:46.503001 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0806 05:00:46.504793 (+ 1792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:00:46.505629 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:00:46.506968 (+ 1339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:00:46.507506 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:00:46.507650 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:00:46.512142 (+ 4492us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 05:00:46.512156 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 05:00:46.512172 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 05:00:46.512217 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0806 05:00:46.515246 (+ 3029us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 05:00:46.515989 (+ 743us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 05:00:46.516002 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 05:00:46.516004 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 05:00:46.516170 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 05:00:46.516591 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 05:00:46.516596 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 05:00:46.516597 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 05:00:46.516961 (+ 364us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 05:00:46.517338 (+ 377us) server_negotiation.cc:1035] Waiting for connection context
0806 05:00:46.517622 (+ 284us) server_negotiation.cc:299] Negotiation successful
0806 05:00:46.517847 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260806 05:00:46.518998 31310 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 05:00:46.556273 31335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:00:46.555828 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.65:38017 (local address 127.0.0.1:54352)
0806 05:00:46.556029 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0806 05:00:46.556111 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.96.65:38017: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
I20260806 05:00:46.557403 31335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:00:46.556978 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.65:38017 (local address 127.0.0.1:54360)
0806 05:00:46.557178 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0806 05:00:46.557243 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.96.65:38017: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":92}
I20260806 05:00:46.640872 9601 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu with pid 31117
I20260806 05:00:46.669960 9601 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu pid 30555
I20260806 05:00:46.671084 30651 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 05:00:46.781256 31335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:00:46.780525 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.65:38017 (local address 127.0.0.1:54368)
0806 05:00:46.780769 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0806 05:00:46.780898 (+ 129us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.96.65:38017: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
I20260806 05:00:46.781563 9601 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu with pid 30555
I20260806 05:00:46.808300 9601 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
/tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785992227050689-9601-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785992227050689-9601-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785992227050689-9601-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785992227050689-9601-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.96.126:39911
--webserver_interface=127.9.96.126
--webserver_port=40163
--keytab_file=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.126.keytab
--principal=kudu/127.9.96.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.96.84:40741
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.96.126:39911
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785992227050689-9601-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785992227050689-9601-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785992227050689-9601-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 05:00:47.157862 31411 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 05:00:47.158136 31411 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 05:00:47.158234 31411 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 05:00:47.165385 31411 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 05:00:47.165488 31411 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 05:00:47.165536 31411 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 05:00:47.165602 31411 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 05:00:47.165647 31411 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 05:00:47.175084 31411 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.96.84:40741
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785992227050689-9601-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785992227050689-9601-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.9.96.126:39911
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.126.keytab
--principal=kudu/127.9.96.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.96.126:39911
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785992227050689-9601-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.96.126
--webserver_port=40163
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785992227050689-9601-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 03:56:12 UTC on ccycloud.quasar-guijbm.root.comops.site
build id 4569618
ASAN enabled
I20260806 05:00:47.176322 31411 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 05:00:47.177523 31411 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 05:00:47.209657 31423 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 05:00:47.209621 31418 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 05:00:47.209709 31417 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 05:00:47.210949 31411 server_base.cc:1034] running on GCE node
I20260806 05:00:47.212107 31411 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 05:00:47.213840 31411 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 05:00:47.215099 31411 hybrid_clock.cc:648] HybridClock initialized: now 1785992447215029 us; error 52 us; skew 500 ppm
I20260806 05:00:47.215606 31411 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 05:00:47 dist-test-slave-x4qh krb5kdc[30534](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992447, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.96.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 05:00:47.223616 31411 init.cc:377] Logged in from keytab as kudu/127.9.96.126@KRBTEST.COM (short username kudu)
I20260806 05:00:47.225507 31411 webserver.cc:460] Webserver started at http://127.9.96.126:40163/ using document root <none> and password file <none>
I20260806 05:00:47.226331 31411 fs_manager.cc:359] Metadata directory not provided
I20260806 05:00:47.226511 31411 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 05:00:47.233000 31411 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.004s sys 0.000s
I20260806 05:00:47.236223 31434 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 05:00:47.237557 31411 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260806 05:00:47.237697 31411 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785992227050689-9601-0/minicluster-data/master-0/data,/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785992227050689-9601-0/minicluster-data/master-0/wal
uuid: "245b5b6f6eca4e65b1c4e9298645efec"
format_stamp: "Formatted at 2026-08-06 05:00:44 on dist-test-slave-x4qh"
server_key: "2aded742f873db7f2e686dc6a545a650"
server_key_iv: "0de062344079dc557791a2862f33c17a"
server_key_version: "encryptionkey@0"
I20260806 05:00:47.238842 31411 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785992227050689-9601-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785992227050689-9601-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785992227050689-9601-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 05:00:47.251181 31411 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 05:00:47.255168 31411 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 05:00:47.255424 31411 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 05:00:47.275125 31411 rpc_server.cc:307] RPC server started. Bound to: 127.9.96.126:39911
I20260806 05:00:47.275169 31511 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.96.126:39911 every 8 connection(s)
I20260806 05:00:47.276732 31411 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785992227050689-9601-0/minicluster-data/master-0/data/info.pb
I20260806 05:00:47.283731 31513 sys_catalog.cc:263] Verifying existing consensus state
I20260806 05:00:47.284994 9601 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu as pid 31411
I20260806 05:00:47.285837 9601 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
/tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785992227050689-9601-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785992227050689-9601-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785992227050689-9601-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785992227050689-9601-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.96.65:38017
--local_ip_for_outbound_sockets=127.9.96.65
--tserver_master_addrs=127.9.96.126:39911
--webserver_port=43473
--webserver_interface=127.9.96.65
--keytab_file=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.65.keytab
--principal=kudu/127.9.96.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.96.84:40741
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785992227050689-9601-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785992227050689-9601-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785992227050689-9601-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 05:00:47.287212 31513 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 245b5b6f6eca4e65b1c4e9298645efec: Bootstrap starting.
I20260806 05:00:47.310699 31513 log.cc:824] T 00000000000000000000000000000000 P 245b5b6f6eca4e65b1c4e9298645efec: Log is configured to *not* fsync() on all Append() calls
I20260806 05:00:47.327498 31513 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 245b5b6f6eca4e65b1c4e9298645efec: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 05:00:47.328145 31513 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 245b5b6f6eca4e65b1c4e9298645efec: Bootstrap complete.
I20260806 05:00:47.341810 31513 raft_consensus.cc:348] T 00000000000000000000000000000000 P 245b5b6f6eca4e65b1c4e9298645efec [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "245b5b6f6eca4e65b1c4e9298645efec" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 39911 } }
I20260806 05:00:47.343009 31513 raft_consensus.cc:729] T 00000000000000000000000000000000 P 245b5b6f6eca4e65b1c4e9298645efec [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 245b5b6f6eca4e65b1c4e9298645efec, State: Initialized, Role: FOLLOWER
I20260806 05:00:47.343743 31513 consensus_queue.cc:260] T 00000000000000000000000000000000 P 245b5b6f6eca4e65b1c4e9298645efec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "245b5b6f6eca4e65b1c4e9298645efec" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 39911 } }
I20260806 05:00:47.344048 31513 raft_consensus.cc:388] T 00000000000000000000000000000000 P 245b5b6f6eca4e65b1c4e9298645efec [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 05:00:47.344177 31513 raft_consensus.cc:482] T 00000000000000000000000000000000 P 245b5b6f6eca4e65b1c4e9298645efec [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 05:00:47.344350 31513 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 245b5b6f6eca4e65b1c4e9298645efec [term 1 FOLLOWER]: Advancing to term 2
I20260806 05:00:47.350112 31513 raft_consensus.cc:504] T 00000000000000000000000000000000 P 245b5b6f6eca4e65b1c4e9298645efec [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "245b5b6f6eca4e65b1c4e9298645efec" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 39911 } }
I20260806 05:00:47.350739 31513 leader_election.cc:302] T 00000000000000000000000000000000 P 245b5b6f6eca4e65b1c4e9298645efec [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 245b5b6f6eca4e65b1c4e9298645efec; no voters:
I20260806 05:00:47.351583 31513 leader_election.cc:288] T 00000000000000000000000000000000 P 245b5b6f6eca4e65b1c4e9298645efec [CANDIDATE]: Term 2 election: Requested vote from peers
I20260806 05:00:47.351755 31518 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 245b5b6f6eca4e65b1c4e9298645efec [term 2 FOLLOWER]: Leader election won for term 2
I20260806 05:00:47.353155 31518 raft_consensus.cc:686] T 00000000000000000000000000000000 P 245b5b6f6eca4e65b1c4e9298645efec [term 2 LEADER]: Becoming Leader. State: Replica: 245b5b6f6eca4e65b1c4e9298645efec, State: Running, Role: LEADER
I20260806 05:00:47.353685 31518 consensus_queue.cc:237] T 00000000000000000000000000000000 P 245b5b6f6eca4e65b1c4e9298645efec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "245b5b6f6eca4e65b1c4e9298645efec" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 39911 } }
I20260806 05:00:47.354023 31513 sys_catalog.cc:564] T 00000000000000000000000000000000 P 245b5b6f6eca4e65b1c4e9298645efec [sys.catalog]: configured and running, proceeding with master startup.
I20260806 05:00:47.356371 31519 sys_catalog.cc:455] T 00000000000000000000000000000000 P 245b5b6f6eca4e65b1c4e9298645efec [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "245b5b6f6eca4e65b1c4e9298645efec" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "245b5b6f6eca4e65b1c4e9298645efec" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 39911 } } }
I20260806 05:00:47.356364 31520 sys_catalog.cc:455] T 00000000000000000000000000000000 P 245b5b6f6eca4e65b1c4e9298645efec [sys.catalog]: SysCatalogTable state changed. Reason: New leader 245b5b6f6eca4e65b1c4e9298645efec. Latest consensus state: current_term: 2 leader_uuid: "245b5b6f6eca4e65b1c4e9298645efec" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "245b5b6f6eca4e65b1c4e9298645efec" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 39911 } } }
I20260806 05:00:47.356827 31519 sys_catalog.cc:458] T 00000000000000000000000000000000 P 245b5b6f6eca4e65b1c4e9298645efec [sys.catalog]: This master's current role is: LEADER
I20260806 05:00:47.356914 31520 sys_catalog.cc:458] T 00000000000000000000000000000000 P 245b5b6f6eca4e65b1c4e9298645efec [sys.catalog]: This master's current role is: LEADER
I20260806 05:00:47.361234 31524 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 05:00:47.369452 31524 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 05:00:47.370954 31524 catalog_manager.cc:1259] Loaded cluster ID: c694851103804b2c91a29a7c087b7b2a
I20260806 05:00:47.371114 31524 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 05:00:47.374426 31524 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260806 05:00:47.392529 31524 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260806 05:00:47.402503 31541 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260806 05:00:47.406769 31524 catalog_manager.cc:1504] Loading token signing keys...
I20260806 05:00:47.408527 31524 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 245b5b6f6eca4e65b1c4e9298645efec: Loaded TSK: 0
I20260806 05:00:47.410061 31524 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260806 05:00:47.665665 31516 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 05:00:47.665972 31516 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 05:00:47.666136 31516 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 05:00:47.673517 31516 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 05:00:47.673662 31516 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 05:00:47.673859 31516 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.96.65
I20260806 05:00:47.683611 31516 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.96.84:40741
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785992227050689-9601-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785992227050689-9601-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.65.keytab
--principal=kudu/127.9.96.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.96.65:38017
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785992227050689-9601-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.96.65
--webserver_port=43473
--tserver_master_addrs=127.9.96.126:39911
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.96.65
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785992227050689-9601-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 03:56:12 UTC on ccycloud.quasar-guijbm.root.comops.site
build id 4569618
ASAN enabled
I20260806 05:00:47.684901 31516 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 05:00:47.686228 31516 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 05:00:47.700619 31548 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 05:00:47.701310 31549 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 05:00:47.701534 31552 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 05:00:47.702018 31516 server_base.cc:1034] running on GCE node
I20260806 05:00:47.702654 31516 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 05:00:47.703677 31516 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 05:00:47.704926 31516 hybrid_clock.cc:648] HybridClock initialized: now 1785992447704880 us; error 49 us; skew 500 ppm
I20260806 05:00:47.705255 31516 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 05:00:47 dist-test-slave-x4qh krb5kdc[30534](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992447, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.96.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 05:00:47.708592 31516 init.cc:377] Logged in from keytab as kudu/127.9.96.65@KRBTEST.COM (short username kudu)
I20260806 05:00:47.710214 31516 webserver.cc:460] Webserver started at http://127.9.96.65:43473/ using document root <none> and password file <none>
I20260806 05:00:47.710932 31516 fs_manager.cc:359] Metadata directory not provided
I20260806 05:00:47.711081 31516 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 05:00:47.716480 31516 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260806 05:00:47.719462 31564 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 05:00:47.720906 31516 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260806 05:00:47.721148 31516 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785992227050689-9601-0/minicluster-data/ts-0/data,/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785992227050689-9601-0/minicluster-data/ts-0/wal
uuid: "7bf95bbc6b8240e4b043b56c9cf673dc"
format_stamp: "Formatted at 2026-08-06 05:00:44 on dist-test-slave-x4qh"
server_key: "efa1633140f1902db8c8b3c77b52919e"
server_key_iv: "e525cdafc2ca69de881c87c31504b8ae"
server_key_version: "encryptionkey@0"
I20260806 05:00:47.722158 31516 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785992227050689-9601-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785992227050689-9601-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785992227050689-9601-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 05:00:47.740856 31516 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 05:00:47.745095 31516 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 05:00:47.745374 31516 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 05:00:47.746323 31516 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 05:00:47.748154 31577 ts_tablet_manager.cc:542] Loading tablet metadata (0/1 complete)
W20260806 05:00:47.753439 31577 ts_tablet_manager.cc:1852] T 248caed6579e49e8992e2265dd991800 P 7bf95bbc6b8240e4b043b56c9cf673dc: Tablet Manager startup: Rolling forward tablet deletion of type TABLET_DATA_DELETED
I20260806 05:00:47.754472 31577 ts_tablet_manager.cc:1905] T 248caed6579e49e8992e2265dd991800 P 7bf95bbc6b8240e4b043b56c9cf673dc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 05:00:47.761598 31577 ts_tablet_manager.cc:1918] T 248caed6579e49e8992e2265dd991800 P 7bf95bbc6b8240e4b043b56c9cf673dc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 05:00:47.761852 31577 log.cc:1196] T 248caed6579e49e8992e2265dd991800 P 7bf95bbc6b8240e4b043b56c9cf673dc: Deleting WAL directory at /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785992227050689-9601-0/minicluster-data/ts-0/wal/wals/248caed6579e49e8992e2265dd991800
I20260806 05:00:47.762400 31577 ts_tablet_manager.cc:1939] T 248caed6579e49e8992e2265dd991800 P 7bf95bbc6b8240e4b043b56c9cf673dc: Deleting consensus metadata
I20260806 05:00:47.762791 31516 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 0 live tablets)
I20260806 05:00:47.762894 31516 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.015s user 0.001s sys 0.000s
I20260806 05:00:47.762987 31516 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 05:00:47.763053 31516 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 05:00:47.796795 31516 rpc_server.cc:307] RPC server started. Bound to: 127.9.96.65:38017
I20260806 05:00:47.796844 31736 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.96.65:38017 every 8 connection(s)
I20260806 05:00:47.798296 31516 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785992227050689-9601-0/minicluster-data/ts-0/data/info.pb
I20260806 05:00:47.799453 9601 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu as pid 31516
I20260806 05:00:47.800104 9601 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
/tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785992227050689-9601-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785992227050689-9601-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785992227050689-9601-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785992227050689-9601-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.96.66:43613
--local_ip_for_outbound_sockets=127.9.96.66
--tserver_master_addrs=127.9.96.126:39911
--webserver_port=45935
--webserver_interface=127.9.96.66
--keytab_file=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.66.keytab
--principal=kudu/127.9.96.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.96.84:40741
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785992227050689-9601-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785992227050689-9601-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785992227050689-9601-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 05:00:47 dist-test-slave-x4qh krb5kdc[30534](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992447, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.96.65@KRBTEST.COM for kudu/127.9.96.126@KRBTEST.COM
I20260806 05:00:47.824622 31742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:00:47.801399 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:39911 (local address 127.9.96.65:58269)
0806 05:00:47.802459 (+ 1060us) negotiation.cc:107] Waiting for socket to connect
0806 05:00:47.802512 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0806 05:00:47.805292 (+ 2780us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 05:00:47.806676 (+ 1384us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 05:00:47.806689 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 05:00:47.807531 (+ 842us) client_negotiation.cc:189] Negotiated authn=SASL
0806 05:00:47.808285 (+ 754us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 05:00:47.808324 (+ 39us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 05:00:47.809966 (+ 1642us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 05:00:47.809975 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 05:00:47.811495 (+ 1520us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 05:00:47.811508 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 05:00:47.813003 (+ 1495us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 05:00:47.813008 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 05:00:47.813186 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:00:47.813878 (+ 692us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 05:00:47.813899 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0806 05:00:47.815990 (+ 2091us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 05:00:47.820662 (+ 4672us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 05:00:47.820669 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 05:00:47.820679 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0806 05:00:47.821073 (+ 394us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 05:00:47.821606 (+ 533us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 05:00:47.821609 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 05:00:47.821611 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 05:00:47.821766 (+ 155us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 05:00:47.822705 (+ 939us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 05:00:47.822710 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 05:00:47.822971 (+ 261us) client_negotiation.cc:769] Sending connection context
0806 05:00:47.823181 (+ 210us) client_negotiation.cc:240] Negotiation successful
0806 05:00:47.823664 (+ 483us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":757,"thread_start_us":554,"threads_started":1}
I20260806 05:00:47.825834 31739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:00:47.801347 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.96.65:58269 (local address 127.9.96.126:39911)
0806 05:00:47.802114 (+ 767us) server_negotiation.cc:206] Beginning negotiation
0806 05:00:47.802131 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0806 05:00:47.802651 (+ 520us) server_negotiation.cc:407] Connection header received
0806 05:00:47.805860 (+ 3209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 05:00:47.805873 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 05:00:47.806181 (+ 308us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 05:00:47.806457 (+ 276us) server_negotiation.cc:226] Negotiated authn=SASL
0806 05:00:47.808554 (+ 2097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:00:47.809778 (+ 1224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:00:47.811642 (+ 1864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:00:47.812789 (+ 1147us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:00:47.812994 (+ 205us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:00:47.816295 (+ 3301us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 05:00:47.816307 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 05:00:47.816338 (+ 31us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 05:00:47.816393 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0806 05:00:47.820510 (+ 4117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 05:00:47.821240 (+ 730us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 05:00:47.821253 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 05:00:47.821264 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0806 05:00:47.821492 (+ 228us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 05:00:47.821888 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 05:00:47.821896 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 05:00:47.821902 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 05:00:47.822570 (+ 668us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 05:00:47.822768 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0806 05:00:47.823529 (+ 761us) server_negotiation.cc:299] Negotiation successful
0806 05:00:47.824003 (+ 474us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":474,"thread_start_us":279,"threads_started":1}
I20260806 05:00:47.828974 31737 heartbeater.cc:347] Connected to a master server at 127.9.96.126:39911
I20260806 05:00:47.829284 31737 heartbeater.cc:464] Registering TS with master...
I20260806 05:00:47.829963 31737 heartbeater.cc:511] Master 127.9.96.126:39911 requested a full tablet report, sending...
I20260806 05:00:47.832533 31461 ts_manager.cc:194] Registered new tserver with Master: 7bf95bbc6b8240e4b043b56c9cf673dc (127.9.96.65:38017)
I20260806 05:00:47.834537 31461 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.96.65@KRBTEST.COM'} at 127.9.96.65:58269
W20260806 05:00:48.138972 31744 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 05:00:48.139243 31744 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 05:00:48.139339 31744 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 05:00:48.146839 31744 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 05:00:48.146950 31744 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 05:00:48.147063 31744 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.96.66
I20260806 05:00:48.156795 31744 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.96.84:40741
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785992227050689-9601-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785992227050689-9601-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.66.keytab
--principal=kudu/127.9.96.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.96.66:43613
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785992227050689-9601-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.96.66
--webserver_port=45935
--tserver_master_addrs=127.9.96.126:39911
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.96.66
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785992227050689-9601-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 03:56:12 UTC on ccycloud.quasar-guijbm.root.comops.site
build id 4569618
ASAN enabled
I20260806 05:00:48.157968 31744 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 05:00:48.159194 31744 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 05:00:48.172360 31750 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 05:00:48.172402 31751 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 05:00:48.172313 31753 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 05:00:48.173280 31744 server_base.cc:1034] running on GCE node
I20260806 05:00:48.173910 31744 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 05:00:48.174825 31744 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 05:00:48.176043 31744 hybrid_clock.cc:648] HybridClock initialized: now 1785992448175987 us; error 44 us; skew 500 ppm
I20260806 05:00:48.176347 31744 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 05:00:48 dist-test-slave-x4qh krb5kdc[30534](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992448, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.96.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 05:00:48.179454 31744 init.cc:377] Logged in from keytab as kudu/127.9.96.66@KRBTEST.COM (short username kudu)
I20260806 05:00:48.180805 31744 webserver.cc:460] Webserver started at http://127.9.96.66:45935/ using document root <none> and password file <none>
I20260806 05:00:48.181479 31744 fs_manager.cc:359] Metadata directory not provided
I20260806 05:00:48.181643 31744 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 05:00:48.187245 31744 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.001s
I20260806 05:00:48.189877 31763 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 05:00:48.191121 31744 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 05:00:48.191300 31744 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785992227050689-9601-0/minicluster-data/ts-1/data,/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785992227050689-9601-0/minicluster-data/ts-1/wal
uuid: "2f286e3a16b847239e12defeff122bec"
format_stamp: "Formatted at 2026-08-06 05:00:45 on dist-test-slave-x4qh"
server_key: "5b9483aab78a2362524a22fd7fa7400b"
server_key_iv: "d6f1132ffd9b9249b72b90a77a248723"
server_key_version: "encryptionkey@0"
I20260806 05:00:48.192327 31744 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785992227050689-9601-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785992227050689-9601-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785992227050689-9601-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 05:00:48.215886 31744 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 05:00:48.219789 31744 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 05:00:48.220075 31744 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 05:00:48.221031 31744 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 05:00:48.222329 31744 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 05:00:48.222460 31744 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 05:00:48.222584 31744 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 05:00:48.222678 31744 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 05:00:48.256837 31744 rpc_server.cc:307] RPC server started. Bound to: 127.9.96.66:43613
I20260806 05:00:48.256934 31926 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.96.66:43613 every 8 connection(s)
I20260806 05:00:48.258256 31744 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785992227050689-9601-0/minicluster-data/ts-1/data/info.pb
I20260806 05:00:48.260514 9601 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu as pid 31744
I20260806 05:00:48.261199 9601 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
/tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785992227050689-9601-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785992227050689-9601-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785992227050689-9601-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785992227050689-9601-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.96.67:41983
--local_ip_for_outbound_sockets=127.9.96.67
--tserver_master_addrs=127.9.96.126:39911
--webserver_port=37687
--webserver_interface=127.9.96.67
--keytab_file=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.67.keytab
--principal=kudu/127.9.96.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.96.84:40741
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785992227050689-9601-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785992227050689-9601-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785992227050689-9601-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 05:00:48 dist-test-slave-x4qh krb5kdc[30534](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992448, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.96.66@KRBTEST.COM for kudu/127.9.96.126@KRBTEST.COM
I20260806 05:00:48.280746 31739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:00:48.260772 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.96.66:38933 (local address 127.9.96.126:39911)
0806 05:00:48.260990 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0806 05:00:48.260996 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 05:00:48.262407 (+ 1411us) server_negotiation.cc:407] Connection header received
0806 05:00:48.265281 (+ 2874us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 05:00:48.265285 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 05:00:48.265354 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 05:00:48.265444 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0806 05:00:48.267600 (+ 2156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:00:48.268281 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:00:48.270704 (+ 2423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:00:48.271121 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:00:48.271210 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:00:48.274807 (+ 3597us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 05:00:48.274812 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 05:00:48.274816 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 05:00:48.274846 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0806 05:00:48.277787 (+ 2941us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 05:00:48.278504 (+ 717us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 05:00:48.278510 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 05:00:48.278512 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 05:00:48.278643 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 05:00:48.279088 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 05:00:48.279092 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 05:00:48.279094 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 05:00:48.279359 (+ 265us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 05:00:48.279563 (+ 204us) server_negotiation.cc:1035] Waiting for connection context
0806 05:00:48.280195 (+ 632us) server_negotiation.cc:299] Negotiation successful
0806 05:00:48.280421 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260806 05:00:48.281685 31929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:00:48.261251 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:39911 (local address 127.9.96.66:38933)
0806 05:00:48.262191 (+ 940us) negotiation.cc:107] Waiting for socket to connect
0806 05:00:48.262263 (+ 72us) client_negotiation.cc:174] Beginning negotiation
0806 05:00:48.264979 (+ 2716us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 05:00:48.265765 (+ 786us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 05:00:48.265777 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 05:00:48.266564 (+ 787us) client_negotiation.cc:189] Negotiated authn=SASL
0806 05:00:48.267413 (+ 849us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 05:00:48.267442 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 05:00:48.268436 (+ 994us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 05:00:48.268444 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 05:00:48.270540 (+ 2096us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 05:00:48.270558 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 05:00:48.271352 (+ 794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 05:00:48.271362 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 05:00:48.271730 (+ 368us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:00:48.272428 (+ 698us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 05:00:48.272451 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0806 05:00:48.274595 (+ 2144us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 05:00:48.277941 (+ 3346us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 05:00:48.277949 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 05:00:48.277963 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0806 05:00:48.278369 (+ 406us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 05:00:48.278759 (+ 390us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 05:00:48.278763 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 05:00:48.278767 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 05:00:48.278960 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 05:00:48.279540 (+ 580us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 05:00:48.279546 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 05:00:48.279841 (+ 295us) client_negotiation.cc:769] Sending connection context
0806 05:00:48.280079 (+ 238us) client_negotiation.cc:240] Negotiation successful
0806 05:00:48.280549 (+ 470us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":605,"thread_start_us":343,"threads_started":1}
I20260806 05:00:48.283098 31927 heartbeater.cc:347] Connected to a master server at 127.9.96.126:39911
I20260806 05:00:48.283490 31927 heartbeater.cc:464] Registering TS with master...
I20260806 05:00:48.284180 31927 heartbeater.cc:511] Master 127.9.96.126:39911 requested a full tablet report, sending...
I20260806 05:00:48.285624 31461 ts_manager.cc:194] Registered new tserver with Master: 2f286e3a16b847239e12defeff122bec (127.9.96.66:43613)
I20260806 05:00:48.286417 31461 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.96.66@KRBTEST.COM'} at 127.9.96.66:38933
W20260806 05:00:48.603044 31931 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 05:00:48.603312 31931 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 05:00:48.603541 31931 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 05:00:48.610780 31931 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 05:00:48.610877 31931 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 05:00:48.611055 31931 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.96.67
I20260806 05:00:48.620469 31931 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.96.84:40741
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785992227050689-9601-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785992227050689-9601-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.67.keytab
--principal=kudu/127.9.96.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.96.67:41983
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785992227050689-9601-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.96.67
--webserver_port=37687
--tserver_master_addrs=127.9.96.126:39911
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.96.67
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785992227050689-9601-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 03:56:12 UTC on ccycloud.quasar-guijbm.root.comops.site
build id 4569618
ASAN enabled
I20260806 05:00:48.621748 31931 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 05:00:48.623003 31931 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 05:00:48.636507 31937 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 05:00:48.636605 31939 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 05:00:48.636580 31944 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 05:00:48.637327 31931 server_base.cc:1034] running on GCE node
I20260806 05:00:48.637914 31931 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 05:00:48.638797 31931 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 05:00:48.639995 31931 hybrid_clock.cc:648] HybridClock initialized: now 1785992448639951 us; error 47 us; skew 500 ppm
I20260806 05:00:48.640250 31931 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 05:00:48 dist-test-slave-x4qh krb5kdc[30534](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992448, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.96.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 05:00:48.643671 31931 init.cc:377] Logged in from keytab as kudu/127.9.96.67@KRBTEST.COM (short username kudu)
I20260806 05:00:48.645125 31931 webserver.cc:460] Webserver started at http://127.9.96.67:37687/ using document root <none> and password file <none>
I20260806 05:00:48.645794 31931 fs_manager.cc:359] Metadata directory not provided
I20260806 05:00:48.645959 31931 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 05:00:48.651024 31931 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260806 05:00:48.653767 31957 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 05:00:48.655166 31931 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260806 05:00:48.655349 31931 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785992227050689-9601-0/minicluster-data/ts-2/data,/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785992227050689-9601-0/minicluster-data/ts-2/wal
uuid: "9e35e3efb1eb4b289f33931312c99bfb"
format_stamp: "Formatted at 2026-08-06 05:00:45 on dist-test-slave-x4qh"
server_key: "12f5b45d70c62786077f37941913201a"
server_key_iv: "a9a117ada6701acebc9a8fac6fecd23e"
server_key_version: "encryptionkey@0"
I20260806 05:00:48.656414 31931 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785992227050689-9601-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785992227050689-9601-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785992227050689-9601-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 05:00:48.673059 31931 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 05:00:48.676802 31931 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 05:00:48.677089 31931 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 05:00:48.678040 31931 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 05:00:48.679303 31931 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 05:00:48.679466 31931 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 05:00:48.679590 31931 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 05:00:48.679683 31931 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 05:00:48.713827 31931 rpc_server.cc:307] RPC server started. Bound to: 127.9.96.67:41983
I20260806 05:00:48.713901 32122 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.96.67:41983 every 8 connection(s)
I20260806 05:00:48.715250 31931 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785992227050689-9601-0/minicluster-data/ts-2/data/info.pb
I20260806 05:00:48.722470 9601 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu as pid 31931
Aug 06 05:00:48 dist-test-slave-x4qh krb5kdc[30534](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992448, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.96.67@KRBTEST.COM for kudu/127.9.96.126@KRBTEST.COM
I20260806 05:00:48.739218 31739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:00:48.717992 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.96.67:51701 (local address 127.9.96.126:39911)
0806 05:00:48.718214 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0806 05:00:48.718218 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 05:00:48.719268 (+ 1050us) server_negotiation.cc:407] Connection header received
0806 05:00:48.721142 (+ 1874us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 05:00:48.721148 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 05:00:48.721240 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 05:00:48.721355 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0806 05:00:48.723198 (+ 1843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:00:48.724323 (+ 1125us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:00:48.726756 (+ 2433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:00:48.727586 (+ 830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:00:48.727719 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:00:48.731997 (+ 4278us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 05:00:48.732004 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 05:00:48.732012 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 05:00:48.732052 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0806 05:00:48.735778 (+ 3726us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 05:00:48.736591 (+ 813us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 05:00:48.736597 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 05:00:48.736601 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 05:00:48.736769 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 05:00:48.737182 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 05:00:48.737210 (+ 28us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 05:00:48.737217 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 05:00:48.737587 (+ 370us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 05:00:48.737694 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0806 05:00:48.738623 (+ 929us) server_negotiation.cc:299] Negotiation successful
0806 05:00:48.738792 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":61,"server-negotiator.queue_time_us":83}
I20260806 05:00:48.740433 32128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:00:48.718376 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:39911 (local address 127.9.96.67:51701)
0806 05:00:48.719082 (+ 706us) negotiation.cc:107] Waiting for socket to connect
0806 05:00:48.719118 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0806 05:00:48.720872 (+ 1754us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 05:00:48.721582 (+ 710us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 05:00:48.721590 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 05:00:48.722181 (+ 591us) client_negotiation.cc:189] Negotiated authn=SASL
0806 05:00:48.722960 (+ 779us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 05:00:48.722985 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 05:00:48.724500 (+ 1515us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 05:00:48.724506 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 05:00:48.726601 (+ 2095us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 05:00:48.726614 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 05:00:48.727786 (+ 1172us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 05:00:48.727791 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 05:00:48.727993 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:00:48.728690 (+ 697us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 05:00:48.728732 (+ 42us) client_negotiation.cc:656] calling sasl_client_start()
0806 05:00:48.731772 (+ 3040us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 05:00:48.735968 (+ 4196us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 05:00:48.735979 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 05:00:48.735998 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0806 05:00:48.736462 (+ 464us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 05:00:48.736887 (+ 425us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 05:00:48.736890 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 05:00:48.736892 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 05:00:48.737069 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 05:00:48.737766 (+ 697us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 05:00:48.737774 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 05:00:48.738222 (+ 448us) client_negotiation.cc:769] Sending connection context
0806 05:00:48.738505 (+ 283us) client_negotiation.cc:240] Negotiation successful
0806 05:00:48.739034 (+ 529us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":467,"thread_start_us":274,"threads_started":1}
I20260806 05:00:48.740466 32130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:00:48.723501 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48786 (local address 127.9.96.126:39911)
0806 05:00:48.724079 (+ 578us) server_negotiation.cc:206] Beginning negotiation
0806 05:00:48.724085 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 05:00:48.724489 (+ 404us) server_negotiation.cc:407] Connection header received
0806 05:00:48.724692 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 05:00:48.724696 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 05:00:48.724750 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 05:00:48.724826 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0806 05:00:48.726763 (+ 1937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:00:48.728043 (+ 1280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:00:48.729930 (+ 1887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:00:48.730579 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:00:48.730748 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:00:48.734069 (+ 3321us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 05:00:48.734076 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 05:00:48.734081 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 05:00:48.734135 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0806 05:00:48.737448 (+ 3313us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 05:00:48.738324 (+ 876us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 05:00:48.738327 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 05:00:48.738329 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 05:00:48.738447 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 05:00:48.738890 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 05:00:48.738895 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 05:00:48.738898 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 05:00:48.739264 (+ 366us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 05:00:48.739622 (+ 358us) server_negotiation.cc:1035] Waiting for connection context
0806 05:00:48.740065 (+ 443us) server_negotiation.cc:299] Negotiation successful
0806 05:00:48.740215 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":23,"server-negotiator.queue_time_us":472,"thread_start_us":342,"threads_started":1}
I20260806 05:00:48.742123 32123 heartbeater.cc:347] Connected to a master server at 127.9.96.126:39911
I20260806 05:00:48.742458 32123 heartbeater.cc:464] Registering TS with master...
I20260806 05:00:48.743245 32123 heartbeater.cc:511] Master 127.9.96.126:39911 requested a full tablet report, sending...
I20260806 05:00:48.744637 31461 ts_manager.cc:194] Registered new tserver with Master: 9e35e3efb1eb4b289f33931312c99bfb (127.9.96.67:41983)
I20260806 05:00:48.745432 31461 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.96.67@KRBTEST.COM'} at 127.9.96.67:51701
I20260806 05:00:48.755316 9601 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 05:00:48.762226 32130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:00:48.756327 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48792 (local address 127.9.96.126:39911)
0806 05:00:48.756511 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0806 05:00:48.756515 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 05:00:48.756996 (+ 481us) server_negotiation.cc:407] Connection header received
0806 05:00:48.757170 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 05:00:48.757174 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 05:00:48.757290 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 05:00:48.757419 (+ 129us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 05:00:48.757800 (+ 381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:00:48.758482 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:00:48.760117 (+ 1635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:00:48.760563 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:00:48.760659 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:00:48.761218 (+ 559us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 05:00:48.761714 (+ 496us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 05:00:48.761817 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0806 05:00:48.761920 (+ 103us) server_negotiation.cc:299] Negotiation successful
0806 05:00:48.761979 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260806 05:00:48.763046 31461 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:48792:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 05:00:48.770341 31461 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 05:00:48.792624 32141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:00:48.781752 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.67:41983 (local address 127.0.0.1:33216)
0806 05:00:48.782868 (+ 1116us) negotiation.cc:107] Waiting for socket to connect
0806 05:00:48.782909 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0806 05:00:48.783529 (+ 620us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 05:00:48.784344 (+ 815us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 05:00:48.784351 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 05:00:48.784403 (+ 52us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 05:00:48.784847 (+ 444us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 05:00:48.784861 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 05:00:48.787139 (+ 2278us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 05:00:48.787144 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 05:00:48.789764 (+ 2620us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 05:00:48.789780 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 05:00:48.791637 (+ 1857us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 05:00:48.791643 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 05:00:48.791834 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:00:48.791896 (+ 62us) client_negotiation.cc:769] Sending connection context
0806 05:00:48.791999 (+ 103us) client_negotiation.cc:240] Negotiation successful
0806 05:00:48.792200 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":817,"mutex_wait_us":3,"thread_start_us":659,"threads_started":1}
I20260806 05:00:48.792809 32147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:00:48.783164 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45144 (local address 127.9.96.66:43613)
0806 05:00:48.783741 (+ 577us) server_negotiation.cc:206] Beginning negotiation
0806 05:00:48.783748 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 05:00:48.784441 (+ 693us) server_negotiation.cc:407] Connection header received
0806 05:00:48.784764 (+ 323us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 05:00:48.784772 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 05:00:48.784990 (+ 218us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 05:00:48.785138 (+ 148us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 05:00:48.786051 (+ 913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:00:48.787569 (+ 1518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:00:48.790103 (+ 2534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:00:48.791426 (+ 1323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:00:48.791525 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:00:48.791746 (+ 221us) server_negotiation.cc:1035] Waiting for connection context
0806 05:00:48.792345 (+ 599us) server_negotiation.cc:299] Negotiation successful
0806 05:00:48.792504 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":399,"thread_start_us":528,"threads_started":1}
I20260806 05:00:48.793594 32149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:00:48.782767 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.66:43613 (local address 127.0.0.1:45144)
0806 05:00:48.784328 (+ 1561us) negotiation.cc:107] Waiting for socket to connect
0806 05:00:48.784347 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0806 05:00:48.784554 (+ 207us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 05:00:48.785398 (+ 844us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 05:00:48.785402 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 05:00:48.785432 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 05:00:48.785861 (+ 429us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 05:00:48.785873 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 05:00:48.787763 (+ 1890us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 05:00:48.787767 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 05:00:48.789931 (+ 2164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 05:00:48.789945 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 05:00:48.791949 (+ 2004us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 05:00:48.791954 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 05:00:48.792132 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:00:48.792162 (+ 30us) client_negotiation.cc:769] Sending connection context
0806 05:00:48.793085 (+ 923us) client_negotiation.cc:240] Negotiation successful
0806 05:00:48.793227 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1427,"threads_started":1}
I20260806 05:00:48.794312 32146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:00:48.782166 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33216 (local address 127.9.96.67:41983)
0806 05:00:48.783005 (+ 839us) server_negotiation.cc:206] Beginning negotiation
0806 05:00:48.783013 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 05:00:48.783075 (+ 62us) server_negotiation.cc:407] Connection header received
0806 05:00:48.783815 (+ 740us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 05:00:48.783831 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 05:00:48.784068 (+ 237us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 05:00:48.785074 (+ 1006us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 05:00:48.785361 (+ 287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:00:48.786989 (+ 1628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:00:48.789989 (+ 3000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:00:48.791398 (+ 1409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:00:48.793642 (+ 2244us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:00:48.793869 (+ 227us) server_negotiation.cc:1035] Waiting for connection context
0806 05:00:48.793942 (+ 73us) server_negotiation.cc:299] Negotiation successful
0806 05:00:48.794065 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":655,"thread_start_us":863,"threads_started":1}
I20260806 05:00:48.798741 32028 tablet_service.cc:1467] Processing CreateTablet for tablet 51bd777d6f1f48a2a126fabf107583ae (DEFAULT_TABLE table=test-table [id=ae901166920d44a5bc085341d84929ea]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 05:00:48.800323 32028 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 51bd777d6f1f48a2a126fabf107583ae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 05:00:48.801671 31841 tablet_service.cc:1467] Processing CreateTablet for tablet 51bd777d6f1f48a2a126fabf107583ae (DEFAULT_TABLE table=test-table [id=ae901166920d44a5bc085341d84929ea]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 05:00:48.803045 31841 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 51bd777d6f1f48a2a126fabf107583ae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 05:00:48.805354 32141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:00:48.783774 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.65:38017 (local address 127.0.0.1:54378)
0806 05:00:48.793117 (+ 9343us) negotiation.cc:107] Waiting for socket to connect
0806 05:00:48.793143 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0806 05:00:48.793237 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 05:00:48.798286 (+ 5049us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 05:00:48.798290 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 05:00:48.798319 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 05:00:48.798474 (+ 155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 05:00:48.798484 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 05:00:48.800802 (+ 2318us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 05:00:48.800807 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 05:00:48.802722 (+ 1915us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 05:00:48.802737 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 05:00:48.804554 (+ 1817us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 05:00:48.804560 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 05:00:48.804770 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:00:48.804809 (+ 39us) client_negotiation.cc:769] Sending connection context
0806 05:00:48.804929 (+ 120us) client_negotiation.cc:240] Negotiation successful
0806 05:00:48.805064 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9261,"thread_start_us":829,"threads_started":1}
I20260806 05:00:48.805619 32150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:00:48.784279 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54378 (local address 127.9.96.65:38017)
0806 05:00:48.797696 (+ 13417us) server_negotiation.cc:206] Beginning negotiation
0806 05:00:48.797704 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 05:00:48.797733 (+ 29us) server_negotiation.cc:407] Connection header received
0806 05:00:48.797824 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 05:00:48.797830 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 05:00:48.798068 (+ 238us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 05:00:48.798272 (+ 204us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 05:00:48.798680 (+ 408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:00:48.800632 (+ 1952us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:00:48.802932 (+ 2300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:00:48.804366 (+ 1434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:00:48.804532 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:00:48.804765 (+ 233us) server_negotiation.cc:1035] Waiting for connection context
0806 05:00:48.804972 (+ 207us) server_negotiation.cc:299] Negotiation successful
0806 05:00:48.805127 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13256,"thread_start_us":14162,"threads_started":1}
I20260806 05:00:48.807451 31648 tablet_service.cc:1467] Processing CreateTablet for tablet 51bd777d6f1f48a2a126fabf107583ae (DEFAULT_TABLE table=test-table [id=ae901166920d44a5bc085341d84929ea]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 05:00:48.808712 31648 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 51bd777d6f1f48a2a126fabf107583ae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 05:00:48.816300 32151 tablet_bootstrap.cc:492] T 51bd777d6f1f48a2a126fabf107583ae P 9e35e3efb1eb4b289f33931312c99bfb: Bootstrap starting.
I20260806 05:00:48.818212 32153 tablet_bootstrap.cc:492] T 51bd777d6f1f48a2a126fabf107583ae P 7bf95bbc6b8240e4b043b56c9cf673dc: Bootstrap starting.
I20260806 05:00:48.819638 32152 tablet_bootstrap.cc:492] T 51bd777d6f1f48a2a126fabf107583ae P 2f286e3a16b847239e12defeff122bec: Bootstrap starting.
I20260806 05:00:48.820864 32151 tablet_bootstrap.cc:654] T 51bd777d6f1f48a2a126fabf107583ae P 9e35e3efb1eb4b289f33931312c99bfb: Neither blocks nor log segments found. Creating new log.
I20260806 05:00:48.821749 32153 tablet_bootstrap.cc:654] T 51bd777d6f1f48a2a126fabf107583ae P 7bf95bbc6b8240e4b043b56c9cf673dc: Neither blocks nor log segments found. Creating new log.
I20260806 05:00:48.822471 32151 log.cc:824] T 51bd777d6f1f48a2a126fabf107583ae P 9e35e3efb1eb4b289f33931312c99bfb: Log is configured to *not* fsync() on all Append() calls
I20260806 05:00:48.822990 32153 log.cc:824] T 51bd777d6f1f48a2a126fabf107583ae P 7bf95bbc6b8240e4b043b56c9cf673dc: Log is configured to *not* fsync() on all Append() calls
I20260806 05:00:48.823994 32152 tablet_bootstrap.cc:654] T 51bd777d6f1f48a2a126fabf107583ae P 2f286e3a16b847239e12defeff122bec: Neither blocks nor log segments found. Creating new log.
I20260806 05:00:48.825064 32152 log.cc:824] T 51bd777d6f1f48a2a126fabf107583ae P 2f286e3a16b847239e12defeff122bec: Log is configured to *not* fsync() on all Append() calls
I20260806 05:00:48.825891 32151 tablet_bootstrap.cc:492] T 51bd777d6f1f48a2a126fabf107583ae P 9e35e3efb1eb4b289f33931312c99bfb: No bootstrap required, opened a new log
I20260806 05:00:48.826205 32151 ts_tablet_manager.cc:1397] T 51bd777d6f1f48a2a126fabf107583ae P 9e35e3efb1eb4b289f33931312c99bfb: Time spent bootstrapping tablet: real 0.010s user 0.001s sys 0.008s
I20260806 05:00:48.826256 32153 tablet_bootstrap.cc:492] T 51bd777d6f1f48a2a126fabf107583ae P 7bf95bbc6b8240e4b043b56c9cf673dc: No bootstrap required, opened a new log
I20260806 05:00:48.826481 32153 ts_tablet_manager.cc:1397] T 51bd777d6f1f48a2a126fabf107583ae P 7bf95bbc6b8240e4b043b56c9cf673dc: Time spent bootstrapping tablet: real 0.008s user 0.007s sys 0.000s
I20260806 05:00:48.827893 32152 tablet_bootstrap.cc:492] T 51bd777d6f1f48a2a126fabf107583ae P 2f286e3a16b847239e12defeff122bec: No bootstrap required, opened a new log
I20260806 05:00:48.828147 32152 ts_tablet_manager.cc:1397] T 51bd777d6f1f48a2a126fabf107583ae P 2f286e3a16b847239e12defeff122bec: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260806 05:00:48.835878 32152 raft_consensus.cc:348] T 51bd777d6f1f48a2a126fabf107583ae P 2f286e3a16b847239e12defeff122bec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e35e3efb1eb4b289f33931312c99bfb" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 41983 } } peers { permanent_uuid: "2f286e3a16b847239e12defeff122bec" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 43613 } } peers { permanent_uuid: "7bf95bbc6b8240e4b043b56c9cf673dc" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 38017 } }
I20260806 05:00:48.836251 32152 raft_consensus.cc:374] T 51bd777d6f1f48a2a126fabf107583ae P 2f286e3a16b847239e12defeff122bec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 05:00:48.836484 32152 raft_consensus.cc:729] T 51bd777d6f1f48a2a126fabf107583ae P 2f286e3a16b847239e12defeff122bec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2f286e3a16b847239e12defeff122bec, State: Initialized, Role: FOLLOWER
I20260806 05:00:48.837167 32152 consensus_queue.cc:260] T 51bd777d6f1f48a2a126fabf107583ae P 2f286e3a16b847239e12defeff122bec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e35e3efb1eb4b289f33931312c99bfb" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 41983 } } peers { permanent_uuid: "2f286e3a16b847239e12defeff122bec" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 43613 } } peers { permanent_uuid: "7bf95bbc6b8240e4b043b56c9cf673dc" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 38017 } }
I20260806 05:00:48.837442 31737 heartbeater.cc:503] Master 127.9.96.126:39911 was elected leader, sending a full tablet report...
I20260806 05:00:48.837304 32151 raft_consensus.cc:348] T 51bd777d6f1f48a2a126fabf107583ae P 9e35e3efb1eb4b289f33931312c99bfb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e35e3efb1eb4b289f33931312c99bfb" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 41983 } } peers { permanent_uuid: "2f286e3a16b847239e12defeff122bec" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 43613 } } peers { permanent_uuid: "7bf95bbc6b8240e4b043b56c9cf673dc" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 38017 } }
I20260806 05:00:48.837311 32153 raft_consensus.cc:348] T 51bd777d6f1f48a2a126fabf107583ae P 7bf95bbc6b8240e4b043b56c9cf673dc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e35e3efb1eb4b289f33931312c99bfb" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 41983 } } peers { permanent_uuid: "2f286e3a16b847239e12defeff122bec" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 43613 } } peers { permanent_uuid: "7bf95bbc6b8240e4b043b56c9cf673dc" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 38017 } }
I20260806 05:00:48.837728 32151 raft_consensus.cc:374] T 51bd777d6f1f48a2a126fabf107583ae P 9e35e3efb1eb4b289f33931312c99bfb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 05:00:48.837735 32153 raft_consensus.cc:374] T 51bd777d6f1f48a2a126fabf107583ae P 7bf95bbc6b8240e4b043b56c9cf673dc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 05:00:48.837824 32151 raft_consensus.cc:729] T 51bd777d6f1f48a2a126fabf107583ae P 9e35e3efb1eb4b289f33931312c99bfb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e35e3efb1eb4b289f33931312c99bfb, State: Initialized, Role: FOLLOWER
I20260806 05:00:48.837832 32153 raft_consensus.cc:729] T 51bd777d6f1f48a2a126fabf107583ae P 7bf95bbc6b8240e4b043b56c9cf673dc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7bf95bbc6b8240e4b043b56c9cf673dc, State: Initialized, Role: FOLLOWER
I20260806 05:00:48.838546 32151 consensus_queue.cc:260] T 51bd777d6f1f48a2a126fabf107583ae P 9e35e3efb1eb4b289f33931312c99bfb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e35e3efb1eb4b289f33931312c99bfb" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 41983 } } peers { permanent_uuid: "2f286e3a16b847239e12defeff122bec" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 43613 } } peers { permanent_uuid: "7bf95bbc6b8240e4b043b56c9cf673dc" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 38017 } }
I20260806 05:00:48.838546 32153 consensus_queue.cc:260] T 51bd777d6f1f48a2a126fabf107583ae P 7bf95bbc6b8240e4b043b56c9cf673dc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e35e3efb1eb4b289f33931312c99bfb" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 41983 } } peers { permanent_uuid: "2f286e3a16b847239e12defeff122bec" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 43613 } } peers { permanent_uuid: "7bf95bbc6b8240e4b043b56c9cf673dc" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 38017 } }
I20260806 05:00:48.840087 32152 ts_tablet_manager.cc:1428] T 51bd777d6f1f48a2a126fabf107583ae P 2f286e3a16b847239e12defeff122bec: Time spent starting tablet: real 0.012s user 0.012s sys 0.000s
I20260806 05:00:48.840543 31927 heartbeater.cc:503] Master 127.9.96.126:39911 was elected leader, sending a full tablet report...
I20260806 05:00:48.844067 32151 ts_tablet_manager.cc:1428] T 51bd777d6f1f48a2a126fabf107583ae P 9e35e3efb1eb4b289f33931312c99bfb: Time spent starting tablet: real 0.018s user 0.014s sys 0.001s
I20260806 05:00:48.844283 32153 ts_tablet_manager.cc:1428] T 51bd777d6f1f48a2a126fabf107583ae P 7bf95bbc6b8240e4b043b56c9cf673dc: Time spent starting tablet: real 0.018s user 0.013s sys 0.004s
I20260806 05:00:48.844494 32123 heartbeater.cc:503] Master 127.9.96.126:39911 was elected leader, sending a full tablet report...
I20260806 05:00:48.868876 32165 raft_consensus.cc:482] T 51bd777d6f1f48a2a126fabf107583ae P 9e35e3efb1eb4b289f33931312c99bfb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 05:00:48.869163 32165 raft_consensus.cc:504] T 51bd777d6f1f48a2a126fabf107583ae P 9e35e3efb1eb4b289f33931312c99bfb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e35e3efb1eb4b289f33931312c99bfb" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 41983 } } peers { permanent_uuid: "2f286e3a16b847239e12defeff122bec" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 43613 } } peers { permanent_uuid: "7bf95bbc6b8240e4b043b56c9cf673dc" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 38017 } }
I20260806 05:00:48.870890 32165 leader_election.cc:288] T 51bd777d6f1f48a2a126fabf107583ae P 9e35e3efb1eb4b289f33931312c99bfb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2f286e3a16b847239e12defeff122bec (127.9.96.66:43613), 7bf95bbc6b8240e4b043b56c9cf673dc (127.9.96.65:38017)
I20260806 05:00:48.878254 32147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:00:48.871400 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.96.67:41791 (local address 127.9.96.66:43613)
0806 05:00:48.871590 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0806 05:00:48.871597 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 05:00:48.871874 (+ 277us) server_negotiation.cc:407] Connection header received
0806 05:00:48.872084 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 05:00:48.872092 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 05:00:48.872176 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 05:00:48.872262 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 05:00:48.872766 (+ 504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:00:48.873790 (+ 1024us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:00:48.876241 (+ 2451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:00:48.877403 (+ 1162us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:00:48.877538 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:00:48.877634 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0806 05:00:48.877905 (+ 271us) server_negotiation.cc:299] Negotiation successful
0806 05:00:48.878007 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260806 05:00:48.878326 32128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:00:48.871253 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.66:43613 (local address 127.9.96.67:41791)
0806 05:00:48.871732 (+ 479us) negotiation.cc:107] Waiting for socket to connect
0806 05:00:48.871748 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 05:00:48.871916 (+ 168us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 05:00:48.872343 (+ 427us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 05:00:48.872349 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 05:00:48.872380 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 05:00:48.872588 (+ 208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 05:00:48.872597 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 05:00:48.873935 (+ 1338us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 05:00:48.873939 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 05:00:48.876089 (+ 2150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 05:00:48.876104 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 05:00:48.877553 (+ 1449us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 05:00:48.877557 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 05:00:48.877682 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:00:48.877739 (+ 57us) client_negotiation.cc:769] Sending connection context
0806 05:00:48.877849 (+ 110us) client_negotiation.cc:240] Negotiation successful
0806 05:00:48.878005 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":50,"mutex_wait_us":72}
I20260806 05:00:48.878674 32150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:00:48.871346 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.96.67:38857 (local address 127.9.96.65:38017)
0806 05:00:48.871557 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0806 05:00:48.871563 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 05:00:48.872396 (+ 833us) server_negotiation.cc:407] Connection header received
0806 05:00:48.872517 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 05:00:48.872521 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 05:00:48.872576 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 05:00:48.872644 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 05:00:48.873181 (+ 537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:00:48.874127 (+ 946us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:00:48.876481 (+ 2354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:00:48.877644 (+ 1163us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:00:48.877748 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:00:48.877844 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0806 05:00:48.878253 (+ 409us) server_negotiation.cc:299] Negotiation successful
0806 05:00:48.878395 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260806 05:00:48.879297 32166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:00:48.871271 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.65:38017 (local address 127.9.96.67:38857)
0806 05:00:48.872263 (+ 992us) negotiation.cc:107] Waiting for socket to connect
0806 05:00:48.872281 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 05:00:48.872411 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 05:00:48.872706 (+ 295us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 05:00:48.872711 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 05:00:48.872741 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 05:00:48.873030 (+ 289us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 05:00:48.873037 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 05:00:48.874275 (+ 1238us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 05:00:48.874285 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 05:00:48.876349 (+ 2064us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 05:00:48.876359 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 05:00:48.877870 (+ 1511us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 05:00:48.877876 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 05:00:48.878034 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:00:48.878063 (+ 29us) client_negotiation.cc:769] Sending connection context
0806 05:00:48.878896 (+ 833us) client_negotiation.cc:240] Negotiation successful
0806 05:00:48.879041 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":861,"mutex_wait_us":167,"thread_start_us":469,"threads_started":1}
I20260806 05:00:48.879483 31870 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "51bd777d6f1f48a2a126fabf107583ae" candidate_uuid: "9e35e3efb1eb4b289f33931312c99bfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2f286e3a16b847239e12defeff122bec" is_pre_election: true
I20260806 05:00:48.880067 31870 raft_consensus.cc:2454] T 51bd777d6f1f48a2a126fabf107583ae P 2f286e3a16b847239e12defeff122bec [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e35e3efb1eb4b289f33931312c99bfb in term 0.
I20260806 05:00:48.880244 31676 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "51bd777d6f1f48a2a126fabf107583ae" candidate_uuid: "9e35e3efb1eb4b289f33931312c99bfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7bf95bbc6b8240e4b043b56c9cf673dc" is_pre_election: true
I20260806 05:00:48.880810 31676 raft_consensus.cc:2454] T 51bd777d6f1f48a2a126fabf107583ae P 7bf95bbc6b8240e4b043b56c9cf673dc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e35e3efb1eb4b289f33931312c99bfb in term 0.
I20260806 05:00:48.880784 31962 leader_election.cc:302] T 51bd777d6f1f48a2a126fabf107583ae P 9e35e3efb1eb4b289f33931312c99bfb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2f286e3a16b847239e12defeff122bec, 9e35e3efb1eb4b289f33931312c99bfb; no voters:
I20260806 05:00:48.881219 32165 raft_consensus.cc:2781] T 51bd777d6f1f48a2a126fabf107583ae P 9e35e3efb1eb4b289f33931312c99bfb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 05:00:48.881389 32165 raft_consensus.cc:482] T 51bd777d6f1f48a2a126fabf107583ae P 9e35e3efb1eb4b289f33931312c99bfb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 05:00:48.881521 32165 raft_consensus.cc:3037] T 51bd777d6f1f48a2a126fabf107583ae P 9e35e3efb1eb4b289f33931312c99bfb [term 0 FOLLOWER]: Advancing to term 1
I20260806 05:00:48.884121 32165 raft_consensus.cc:504] T 51bd777d6f1f48a2a126fabf107583ae P 9e35e3efb1eb4b289f33931312c99bfb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e35e3efb1eb4b289f33931312c99bfb" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 41983 } } peers { permanent_uuid: "2f286e3a16b847239e12defeff122bec" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 43613 } } peers { permanent_uuid: "7bf95bbc6b8240e4b043b56c9cf673dc" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 38017 } }
I20260806 05:00:48.884815 32165 leader_election.cc:288] T 51bd777d6f1f48a2a126fabf107583ae P 9e35e3efb1eb4b289f33931312c99bfb [CANDIDATE]: Term 1 election: Requested vote from peers 2f286e3a16b847239e12defeff122bec (127.9.96.66:43613), 7bf95bbc6b8240e4b043b56c9cf673dc (127.9.96.65:38017)
I20260806 05:00:48.885190 31870 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "51bd777d6f1f48a2a126fabf107583ae" candidate_uuid: "9e35e3efb1eb4b289f33931312c99bfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2f286e3a16b847239e12defeff122bec"
I20260806 05:00:48.885243 31676 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "51bd777d6f1f48a2a126fabf107583ae" candidate_uuid: "9e35e3efb1eb4b289f33931312c99bfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7bf95bbc6b8240e4b043b56c9cf673dc"
I20260806 05:00:48.885394 31870 raft_consensus.cc:3037] T 51bd777d6f1f48a2a126fabf107583ae P 2f286e3a16b847239e12defeff122bec [term 0 FOLLOWER]: Advancing to term 1
I20260806 05:00:48.885434 31676 raft_consensus.cc:3037] T 51bd777d6f1f48a2a126fabf107583ae P 7bf95bbc6b8240e4b043b56c9cf673dc [term 0 FOLLOWER]: Advancing to term 1
I20260806 05:00:48.887634 31676 raft_consensus.cc:2454] T 51bd777d6f1f48a2a126fabf107583ae P 7bf95bbc6b8240e4b043b56c9cf673dc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e35e3efb1eb4b289f33931312c99bfb in term 1.
I20260806 05:00:48.887665 31870 raft_consensus.cc:2454] T 51bd777d6f1f48a2a126fabf107583ae P 2f286e3a16b847239e12defeff122bec [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e35e3efb1eb4b289f33931312c99bfb in term 1.
I20260806 05:00:48.888058 31962 leader_election.cc:302] T 51bd777d6f1f48a2a126fabf107583ae P 9e35e3efb1eb4b289f33931312c99bfb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2f286e3a16b847239e12defeff122bec, 9e35e3efb1eb4b289f33931312c99bfb; no voters:
I20260806 05:00:48.888370 32165 raft_consensus.cc:2781] T 51bd777d6f1f48a2a126fabf107583ae P 9e35e3efb1eb4b289f33931312c99bfb [term 1 FOLLOWER]: Leader election won for term 1
I20260806 05:00:48.888813 32165 raft_consensus.cc:686] T 51bd777d6f1f48a2a126fabf107583ae P 9e35e3efb1eb4b289f33931312c99bfb [term 1 LEADER]: Becoming Leader. State: Replica: 9e35e3efb1eb4b289f33931312c99bfb, State: Running, Role: LEADER
I20260806 05:00:48.889309 32165 consensus_queue.cc:237] T 51bd777d6f1f48a2a126fabf107583ae P 9e35e3efb1eb4b289f33931312c99bfb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e35e3efb1eb4b289f33931312c99bfb" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 41983 } } peers { permanent_uuid: "2f286e3a16b847239e12defeff122bec" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 43613 } } peers { permanent_uuid: "7bf95bbc6b8240e4b043b56c9cf673dc" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 38017 } }
I20260806 05:00:48.893947 31461 catalog_manager.cc:5495] T 51bd777d6f1f48a2a126fabf107583ae P 9e35e3efb1eb4b289f33931312c99bfb reported cstate change: term changed from 0 to 1, leader changed from <none> to 9e35e3efb1eb4b289f33931312c99bfb (127.9.96.67). New cstate: current_term: 1 leader_uuid: "9e35e3efb1eb4b289f33931312c99bfb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e35e3efb1eb4b289f33931312c99bfb" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 41983 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2f286e3a16b847239e12defeff122bec" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 43613 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7bf95bbc6b8240e4b043b56c9cf673dc" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 38017 } health_report { overall_health: UNKNOWN } } }
I20260806 05:00:48.917277 32146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:00:48.912182 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33224 (local address 127.9.96.67:41983)
0806 05:00:48.912417 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0806 05:00:48.912421 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 05:00:48.912439 (+ 18us) server_negotiation.cc:407] Connection header received
0806 05:00:48.912631 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 05:00:48.912634 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 05:00:48.912715 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 05:00:48.912851 (+ 136us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 05:00:48.913175 (+ 324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:00:48.913913 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:00:48.915224 (+ 1311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:00:48.915672 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:00:48.915771 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:00:48.916225 (+ 454us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 05:00:48.916722 (+ 497us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 05:00:48.916829 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0806 05:00:48.916918 (+ 89us) server_negotiation.cc:299] Negotiation successful
0806 05:00:48.916981 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92}
I20260806 05:00:48.925392 31676 raft_consensus.cc:1261] T 51bd777d6f1f48a2a126fabf107583ae P 7bf95bbc6b8240e4b043b56c9cf673dc [term 1 FOLLOWER]: Refusing update from remote peer 9e35e3efb1eb4b289f33931312c99bfb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 05:00:48.925431 31870 raft_consensus.cc:1261] T 51bd777d6f1f48a2a126fabf107583ae P 2f286e3a16b847239e12defeff122bec [term 1 FOLLOWER]: Refusing update from remote peer 9e35e3efb1eb4b289f33931312c99bfb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 05:00:48.926425 32167 consensus_queue.cc:1035] T 51bd777d6f1f48a2a126fabf107583ae P 9e35e3efb1eb4b289f33931312c99bfb [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bf95bbc6b8240e4b043b56c9cf673dc" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 38017 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 05:00:48.926862 32165 consensus_queue.cc:1035] T 51bd777d6f1f48a2a126fabf107583ae P 9e35e3efb1eb4b289f33931312c99bfb [LEADER]: Connected to new peer: Peer: permanent_uuid: "2f286e3a16b847239e12defeff122bec" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 43613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 05:00:48.951704 31461 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:48792:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 05:00:48.952016 31461 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:48792:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 05:00:48.955647 31461 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 245b5b6f6eca4e65b1c4e9298645efec: Sending DeleteTablet for 3 replicas of tablet 51bd777d6f1f48a2a126fabf107583ae
I20260806 05:00:48.956827 31841 tablet_service.cc:1514] Processing DeleteTablet for tablet 51bd777d6f1f48a2a126fabf107583ae with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 05:00:48 UTC) from {username='kudu', principal='kudu/127.9.96.126@KRBTEST.COM'} at 127.0.0.1:45144
I20260806 05:00:48.957147 32028 tablet_service.cc:1514] Processing DeleteTablet for tablet 51bd777d6f1f48a2a126fabf107583ae with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 05:00:48 UTC) from {username='kudu', principal='kudu/127.9.96.126@KRBTEST.COM'} at 127.0.0.1:33216
I20260806 05:00:48.957176 31648 tablet_service.cc:1514] Processing DeleteTablet for tablet 51bd777d6f1f48a2a126fabf107583ae with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 05:00:48 UTC) from {username='kudu', principal='kudu/127.9.96.126@KRBTEST.COM'} at 127.0.0.1:54378
I20260806 05:00:48.958458 9601 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu pid 31516
I20260806 05:00:48.971011 32182 tablet_replica.cc:330] T 51bd777d6f1f48a2a126fabf107583ae P 9e35e3efb1eb4b289f33931312c99bfb: stopping tablet replica
I20260806 05:00:48.971792 32182 raft_consensus.cc:2229] T 51bd777d6f1f48a2a126fabf107583ae P 9e35e3efb1eb4b289f33931312c99bfb [term 1 LEADER]: Raft consensus shutting down.
I20260806 05:00:48.972741 32182 raft_consensus.cc:2258] T 51bd777d6f1f48a2a126fabf107583ae P 9e35e3efb1eb4b289f33931312c99bfb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 05:00:48.975643 32182 ts_tablet_manager.cc:1905] T 51bd777d6f1f48a2a126fabf107583ae P 9e35e3efb1eb4b289f33931312c99bfb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 05:00:48.978123 32181 tablet_replica.cc:330] T 51bd777d6f1f48a2a126fabf107583ae P 7bf95bbc6b8240e4b043b56c9cf673dc: stopping tablet replica
I20260806 05:00:48.978684 32181 raft_consensus.cc:2229] T 51bd777d6f1f48a2a126fabf107583ae P 7bf95bbc6b8240e4b043b56c9cf673dc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 05:00:48.979269 32181 raft_consensus.cc:2258] T 51bd777d6f1f48a2a126fabf107583ae P 7bf95bbc6b8240e4b043b56c9cf673dc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 05:00:48.979859 32180 tablet_replica.cc:330] T 51bd777d6f1f48a2a126fabf107583ae P 2f286e3a16b847239e12defeff122bec: stopping tablet replica
I20260806 05:00:48.980146 32150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:00:48.959762 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54380 (local address 127.9.96.65:38017)
0806 05:00:48.959942 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0806 05:00:48.959948 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 05:00:48.959966 (+ 18us) server_negotiation.cc:407] Connection header received
0806 05:00:48.960022 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 05:00:48.960028 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 05:00:48.960110 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 05:00:48.960216 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0806 05:00:48.962338 (+ 2122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:00:48.963641 (+ 1303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:00:48.966141 (+ 2500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:00:48.966828 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:00:48.966956 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:00:48.971405 (+ 4449us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 05:00:48.971426 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 05:00:48.971445 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 05:00:48.971492 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0806 05:00:48.975608 (+ 4116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 05:00:48.976614 (+ 1006us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 05:00:48.976623 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 05:00:48.976627 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 05:00:48.976822 (+ 195us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 05:00:48.977304 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 05:00:48.977309 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 05:00:48.977313 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 05:00:48.977845 (+ 532us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 05:00:48.978171 (+ 326us) server_negotiation.cc:1035] Waiting for connection context
0806 05:00:48.979481 (+ 1310us) server_negotiation.cc:299] Negotiation successful
0806 05:00:48.979758 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260806 05:00:48.980652 32180 raft_consensus.cc:2229] T 51bd777d6f1f48a2a126fabf107583ae P 2f286e3a16b847239e12defeff122bec [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 05:00:48.981058 31731 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 05:00:48.981287 32180 raft_consensus.cc:2258] T 51bd777d6f1f48a2a126fabf107583ae P 2f286e3a16b847239e12defeff122bec [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 05:00:48.983350 32180 ts_tablet_manager.cc:1905] T 51bd777d6f1f48a2a126fabf107583ae P 2f286e3a16b847239e12defeff122bec: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 05:00:48.984265 32182 ts_tablet_manager.cc:1918] T 51bd777d6f1f48a2a126fabf107583ae P 9e35e3efb1eb4b289f33931312c99bfb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 05:00:48.984490 32182 log.cc:1196] T 51bd777d6f1f48a2a126fabf107583ae P 9e35e3efb1eb4b289f33931312c99bfb: Deleting WAL directory at /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785992227050689-9601-0/minicluster-data/ts-2/wal/wals/51bd777d6f1f48a2a126fabf107583ae
I20260806 05:00:48.985128 32182 ts_tablet_manager.cc:1939] T 51bd777d6f1f48a2a126fabf107583ae P 9e35e3efb1eb4b289f33931312c99bfb: Deleting consensus metadata
I20260806 05:00:48.987032 31437 catalog_manager.cc:4841] TS 9e35e3efb1eb4b289f33931312c99bfb (127.9.96.67:41983): tablet 51bd777d6f1f48a2a126fabf107583ae (table test-table [id=ae901166920d44a5bc085341d84929ea]) successfully deleted
I20260806 05:00:48.991290 32180 ts_tablet_manager.cc:1918] T 51bd777d6f1f48a2a126fabf107583ae P 2f286e3a16b847239e12defeff122bec: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 05:00:48.991494 32180 log.cc:1196] T 51bd777d6f1f48a2a126fabf107583ae P 2f286e3a16b847239e12defeff122bec: Deleting WAL directory at /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785992227050689-9601-0/minicluster-data/ts-1/wal/wals/51bd777d6f1f48a2a126fabf107583ae
I20260806 05:00:48.992009 32180 ts_tablet_manager.cc:1939] T 51bd777d6f1f48a2a126fabf107583ae P 2f286e3a16b847239e12defeff122bec: Deleting consensus metadata
I20260806 05:00:48.993643 31440 catalog_manager.cc:4841] TS 2f286e3a16b847239e12defeff122bec (127.9.96.66:43613): tablet 51bd777d6f1f48a2a126fabf107583ae (table test-table [id=ae901166920d44a5bc085341d84929ea]) successfully deleted
I20260806 05:00:49.078475 9601 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu with pid 31516
I20260806 05:00:49.078835 32181 ts_tablet_manager.cc:1905] T 51bd777d6f1f48a2a126fabf107583ae P 7bf95bbc6b8240e4b043b56c9cf673dc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 05:00:49.109030 9601 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu pid 31744
W20260806 05:00:49.110304 31437 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.9.96.65:38017 (error 108)
I20260806 05:00:49.112385 32141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:00:49.111823 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.65:38017 (local address 127.0.0.1:54390)
0806 05:00:49.112070 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0806 05:00:49.112178 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.96.65:38017: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":136}
W20260806 05:00:49.112646 31437 catalog_manager.cc:4584] TS 7bf95bbc6b8240e4b043b56c9cf673dc (127.9.96.65:38017): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 51bd777d6f1f48a2a126fabf107583ae: Network error: Client connection negotiation failed: client connection to 127.9.96.65:38017: connect: Connection refused (error 111)
I20260806 05:00:49.128108 32147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:00:49.109963 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45146 (local address 127.9.96.66:43613)
0806 05:00:49.110163 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0806 05:00:49.110168 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 05:00:49.113175 (+ 3007us) server_negotiation.cc:407] Connection header received
0806 05:00:49.113385 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 05:00:49.113398 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 05:00:49.113513 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 05:00:49.113601 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0806 05:00:49.115363 (+ 1762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:00:49.116140 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:00:49.118031 (+ 1891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:00:49.118392 (+ 361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:00:49.118482 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:00:49.121840 (+ 3358us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 05:00:49.121853 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 05:00:49.121866 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 05:00:49.121897 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0806 05:00:49.125164 (+ 3267us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 05:00:49.125995 (+ 831us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 05:00:49.126002 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 05:00:49.126004 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 05:00:49.126132 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 05:00:49.126570 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 05:00:49.126574 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 05:00:49.126575 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 05:00:49.126940 (+ 365us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 05:00:49.127139 (+ 199us) server_negotiation.cc:1035] Waiting for connection context
0806 05:00:49.127614 (+ 475us) server_negotiation.cc:299] Negotiation successful
0806 05:00:49.127790 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260806 05:00:49.128777 31922 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 05:00:49.166388 32141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:00:49.165939 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.65:38017 (local address 127.0.0.1:54392)
0806 05:00:49.166156 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0806 05:00:49.166243 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.96.65:38017: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":76}
I20260806 05:00:49.167682 32141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:00:49.167164 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.65:38017 (local address 127.0.0.1:54402)
0806 05:00:49.167344 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0806 05:00:49.167500 (+ 156us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.96.65:38017: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260806 05:00:49.228379 32141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:00:49.227773 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.65:38017 (local address 127.0.0.1:54414)
0806 05:00:49.228051 (+ 278us) negotiation.cc:107] Waiting for socket to connect
0806 05:00:49.228186 (+ 135us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.96.65:38017: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":86}
I20260806 05:00:49.229676 32141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:00:49.229230 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.65:38017 (local address 127.0.0.1:54426)
0806 05:00:49.229410 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0806 05:00:49.229504 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.96.65:38017: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260806 05:00:49.260947 9601 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu with pid 31744
I20260806 05:00:49.293181 9601 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu pid 31931
I20260806 05:00:49.310385 32141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:00:49.309850 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.65:38017 (local address 127.0.0.1:54428)
0806 05:00:49.310093 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0806 05:00:49.310214 (+ 121us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.96.65:38017: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":84}
I20260806 05:00:49.311812 32141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:00:49.311167 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.65:38017 (local address 127.0.0.1:54442)
0806 05:00:49.311577 (+ 410us) negotiation.cc:107] Waiting for socket to connect
0806 05:00:49.311671 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.96.65:38017: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":145}
I20260806 05:00:49.312717 32146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:00:49.294084 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33238 (local address 127.9.96.67:41983)
0806 05:00:49.294285 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0806 05:00:49.294292 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 05:00:49.294646 (+ 354us) server_negotiation.cc:407] Connection header received
0806 05:00:49.294873 (+ 227us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 05:00:49.294879 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 05:00:49.294973 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 05:00:49.295107 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0806 05:00:49.296931 (+ 1824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:00:49.298245 (+ 1314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:00:49.300331 (+ 2086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:00:49.301028 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:00:49.301168 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:00:49.304714 (+ 3546us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 05:00:49.304733 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 05:00:49.304754 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 05:00:49.304805 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0806 05:00:49.309371 (+ 4566us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 05:00:49.310275 (+ 904us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 05:00:49.310283 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 05:00:49.310285 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 05:00:49.310447 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 05:00:49.311045 (+ 598us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 05:00:49.311049 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 05:00:49.311051 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 05:00:49.311509 (+ 458us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 05:00:49.311869 (+ 360us) server_negotiation.cc:1035] Waiting for connection context
0806 05:00:49.312207 (+ 338us) server_negotiation.cc:299] Negotiation successful
0806 05:00:49.312375 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260806 05:00:49.313552 32116 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 05:00:49.434839 9601 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu with pid 31931
I20260806 05:00:49.465296 9601 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu pid 31411
I20260806 05:00:49.466513 31506 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 05:00:49.581127 32141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:00:49.580596 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.65:38017 (local address 127.0.0.1:54452)
0806 05:00:49.580797 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0806 05:00:49.580884 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.96.65:38017: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
I20260806 05:00:49.581161 9601 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu with pid 31411
I20260806 05:00:49.609660 9601 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
/tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785992227050689-9601-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785992227050689-9601-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785992227050689-9601-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785992227050689-9601-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.96.126:39911
--webserver_interface=127.9.96.126
--webserver_port=40163
--keytab_file=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.126.keytab
--principal=kudu/127.9.96.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.96.84:40741
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.96.126:39911
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785992227050689-9601-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785992227050689-9601-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785992227050689-9601-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 05:00:49.945906 32192 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 05:00:49.946174 32192 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 05:00:49.946291 32192 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 05:00:49.953388 32192 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 05:00:49.953486 32192 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 05:00:49.953595 32192 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 05:00:49.953681 32192 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 05:00:49.953749 32192 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 05:00:49.963982 32192 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.96.84:40741
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785992227050689-9601-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785992227050689-9601-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.96.126:39911
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.126.keytab
--principal=kudu/127.9.96.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.96.126:39911
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785992227050689-9601-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.96.126
--webserver_port=40163
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785992227050689-9601-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 03:56:12 UTC on ccycloud.quasar-guijbm.root.comops.site
build id 4569618
ASAN enabled
I20260806 05:00:49.965320 32192 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 05:00:49.966636 32192 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 05:00:49.978310 32203 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 05:00:49.978438 32200 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 05:00:49.978399 32198 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 05:00:49.979538 32192 server_base.cc:1034] running on GCE node
I20260806 05:00:49.980275 32192 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 05:00:49.981642 32192 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 05:00:49.982885 32192 hybrid_clock.cc:648] HybridClock initialized: now 1785992449982815 us; error 57 us; skew 500 ppm
I20260806 05:00:49.983232 32192 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 05:00:49 dist-test-slave-x4qh krb5kdc[30534](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992449, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.96.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 05:00:49.986411 32192 init.cc:377] Logged in from keytab as kudu/127.9.96.126@KRBTEST.COM (short username kudu)
I20260806 05:00:49.988018 32192 webserver.cc:460] Webserver started at http://127.9.96.126:40163/ using document root <none> and password file <none>
I20260806 05:00:49.988705 32192 fs_manager.cc:359] Metadata directory not provided
I20260806 05:00:49.988858 32192 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 05:00:49.994067 32192 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260806 05:00:49.996884 32213 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 05:00:49.998108 32192 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260806 05:00:49.998294 32192 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785992227050689-9601-0/minicluster-data/master-0/data,/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785992227050689-9601-0/minicluster-data/master-0/wal
uuid: "245b5b6f6eca4e65b1c4e9298645efec"
format_stamp: "Formatted at 2026-08-06 05:00:44 on dist-test-slave-x4qh"
server_key: "2aded742f873db7f2e686dc6a545a650"
server_key_iv: "0de062344079dc557791a2862f33c17a"
server_key_version: "encryptionkey@0"
I20260806 05:00:49.999267 32192 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785992227050689-9601-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785992227050689-9601-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785992227050689-9601-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 05:00:50.010637 32192 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 05:00:50.014856 32192 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 05:00:50.015157 32192 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 05:00:50.034708 32192 rpc_server.cc:307] RPC server started. Bound to: 127.9.96.126:39911
I20260806 05:00:50.034749 32290 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.96.126:39911 every 8 connection(s)
I20260806 05:00:50.036309 32192 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785992227050689-9601-0/minicluster-data/master-0/data/info.pb
I20260806 05:00:50.043068 32292 sys_catalog.cc:263] Verifying existing consensus state
I20260806 05:00:50.045699 9601 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu as pid 32192
I20260806 05:00:50.046763 9601 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
/tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785992227050689-9601-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785992227050689-9601-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785992227050689-9601-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785992227050689-9601-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.96.65:38017
--local_ip_for_outbound_sockets=127.9.96.65
--tserver_master_addrs=127.9.96.126:39911
--webserver_port=43473
--webserver_interface=127.9.96.65
--keytab_file=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.65.keytab
--principal=kudu/127.9.96.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.96.84:40741
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785992227050689-9601-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785992227050689-9601-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785992227050689-9601-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 05:00:50.047142 32292 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 245b5b6f6eca4e65b1c4e9298645efec: Bootstrap starting.
I20260806 05:00:50.071029 32292 log.cc:824] T 00000000000000000000000000000000 P 245b5b6f6eca4e65b1c4e9298645efec: Log is configured to *not* fsync() on all Append() calls
I20260806 05:00:50.087757 32292 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 245b5b6f6eca4e65b1c4e9298645efec: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 05:00:50.088256 32292 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 245b5b6f6eca4e65b1c4e9298645efec: Bootstrap complete.
I20260806 05:00:50.097558 32292 raft_consensus.cc:348] T 00000000000000000000000000000000 P 245b5b6f6eca4e65b1c4e9298645efec [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "245b5b6f6eca4e65b1c4e9298645efec" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 39911 } }
I20260806 05:00:50.098429 32292 raft_consensus.cc:729] T 00000000000000000000000000000000 P 245b5b6f6eca4e65b1c4e9298645efec [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 245b5b6f6eca4e65b1c4e9298645efec, State: Initialized, Role: FOLLOWER
I20260806 05:00:50.099144 32292 consensus_queue.cc:260] T 00000000000000000000000000000000 P 245b5b6f6eca4e65b1c4e9298645efec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "245b5b6f6eca4e65b1c4e9298645efec" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 39911 } }
I20260806 05:00:50.099414 32292 raft_consensus.cc:388] T 00000000000000000000000000000000 P 245b5b6f6eca4e65b1c4e9298645efec [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 05:00:50.099565 32292 raft_consensus.cc:482] T 00000000000000000000000000000000 P 245b5b6f6eca4e65b1c4e9298645efec [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 05:00:50.099753 32292 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 245b5b6f6eca4e65b1c4e9298645efec [term 2 FOLLOWER]: Advancing to term 3
I20260806 05:00:50.104527 32292 raft_consensus.cc:504] T 00000000000000000000000000000000 P 245b5b6f6eca4e65b1c4e9298645efec [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "245b5b6f6eca4e65b1c4e9298645efec" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 39911 } }
I20260806 05:00:50.105134 32292 leader_election.cc:302] T 00000000000000000000000000000000 P 245b5b6f6eca4e65b1c4e9298645efec [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 245b5b6f6eca4e65b1c4e9298645efec; no voters:
I20260806 05:00:50.105923 32292 leader_election.cc:288] T 00000000000000000000000000000000 P 245b5b6f6eca4e65b1c4e9298645efec [CANDIDATE]: Term 3 election: Requested vote from peers
I20260806 05:00:50.106047 32299 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 245b5b6f6eca4e65b1c4e9298645efec [term 3 FOLLOWER]: Leader election won for term 3
I20260806 05:00:50.108453 32292 sys_catalog.cc:564] T 00000000000000000000000000000000 P 245b5b6f6eca4e65b1c4e9298645efec [sys.catalog]: configured and running, proceeding with master startup.
I20260806 05:00:50.113175 32299 raft_consensus.cc:686] T 00000000000000000000000000000000 P 245b5b6f6eca4e65b1c4e9298645efec [term 3 LEADER]: Becoming Leader. State: Replica: 245b5b6f6eca4e65b1c4e9298645efec, State: Running, Role: LEADER
I20260806 05:00:50.113818 32299 consensus_queue.cc:237] T 00000000000000000000000000000000 P 245b5b6f6eca4e65b1c4e9298645efec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "245b5b6f6eca4e65b1c4e9298645efec" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 39911 } }
I20260806 05:00:50.116669 32301 sys_catalog.cc:455] T 00000000000000000000000000000000 P 245b5b6f6eca4e65b1c4e9298645efec [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "245b5b6f6eca4e65b1c4e9298645efec" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "245b5b6f6eca4e65b1c4e9298645efec" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 39911 } } }
I20260806 05:00:50.116777 32302 sys_catalog.cc:455] T 00000000000000000000000000000000 P 245b5b6f6eca4e65b1c4e9298645efec [sys.catalog]: SysCatalogTable state changed. Reason: New leader 245b5b6f6eca4e65b1c4e9298645efec. Latest consensus state: current_term: 3 leader_uuid: "245b5b6f6eca4e65b1c4e9298645efec" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "245b5b6f6eca4e65b1c4e9298645efec" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 39911 } } }
I20260806 05:00:50.117084 32301 sys_catalog.cc:458] T 00000000000000000000000000000000 P 245b5b6f6eca4e65b1c4e9298645efec [sys.catalog]: This master's current role is: LEADER
I20260806 05:00:50.117162 32302 sys_catalog.cc:458] T 00000000000000000000000000000000 P 245b5b6f6eca4e65b1c4e9298645efec [sys.catalog]: This master's current role is: LEADER
I20260806 05:00:50.118088 32310 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 05:00:50.136523 32310 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 05:00:50.138123 32310 catalog_manager.cc:1259] Loaded cluster ID: c694851103804b2c91a29a7c087b7b2a
I20260806 05:00:50.138300 32310 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260806 05:00:50.140749 32310 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f5249d669ad google::LogMessage::Fail() at ??:0
@ 0x7f5249d6988a google::LogMessage::SendToLog() at ??:0
@ 0x7f5249d664de google::LogMessage::Flush() at ??:0
@ 0x7f5249d67a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f525f799512 kudu::master::CatalogManager::PrepareForLeadershipTask()::$_6::operator()() at ??:0
@ 0x7f525f796976 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f525f81e610 kudu::master::CatalogManager::ElectedAsLeaderCb()::$_2::operator()() at ??:0
@ 0x7f525f81e439 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f524c9b2345 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f524c9b591c kudu::ThreadPool::CreateThread()::$_2::operator()() at ??:0
@ 0x7f524c9b5799 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f524c990eaf kudu::Thread::SuperviseThread() at ??:0
@ 0x7f524fc6a6db start_thread at ??:0
@ 0x7f5248c0061f clone at ??:0
W20260806 05:00:50.495035 32294 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 05:00:50.495313 32294 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 05:00:50.495487 32294 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 05:00:50.502665 32294 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 05:00:50.502771 32294 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 05:00:50.502895 32294 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.96.65
I20260806 05:00:50.512730 32294 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.96.84:40741
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785992227050689-9601-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785992227050689-9601-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.65.keytab
--principal=kudu/127.9.96.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.96.65:38017
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785992227050689-9601-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.96.65
--webserver_port=43473
--tserver_master_addrs=127.9.96.126:39911
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.96.65
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785992227050689-9601-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 03:56:12 UTC on ccycloud.quasar-guijbm.root.comops.site
build id 4569618
ASAN enabled
I20260806 05:00:50.513934 32294 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 05:00:50.515230 32294 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 05:00:50.528401 32362 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 05:00:50.528439 32363 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 05:00:50.528463 32366 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 05:00:50.530086 32294 server_base.cc:1034] running on GCE node
I20260806 05:00:50.530791 32294 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 05:00:50.531792 32294 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 05:00:50.533007 32294 hybrid_clock.cc:648] HybridClock initialized: now 1785992450532964 us; error 31 us; skew 500 ppm
I20260806 05:00:50.533337 32294 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 05:00:50 dist-test-slave-x4qh krb5kdc[30534](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992450, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.96.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 05:00:50.536602 32294 init.cc:377] Logged in from keytab as kudu/127.9.96.65@KRBTEST.COM (short username kudu)
I20260806 05:00:50.537966 32294 webserver.cc:460] Webserver started at http://127.9.96.65:43473/ using document root <none> and password file <none>
I20260806 05:00:50.538637 32294 fs_manager.cc:359] Metadata directory not provided
I20260806 05:00:50.538748 32294 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 05:00:50.543999 32294 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.001s
I20260806 05:00:50.549338 32375 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 05:00:50.550694 32294 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 05:00:50.550885 32294 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785992227050689-9601-0/minicluster-data/ts-0/data,/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785992227050689-9601-0/minicluster-data/ts-0/wal
uuid: "7bf95bbc6b8240e4b043b56c9cf673dc"
format_stamp: "Formatted at 2026-08-06 05:00:44 on dist-test-slave-x4qh"
server_key: "efa1633140f1902db8c8b3c77b52919e"
server_key_iv: "e525cdafc2ca69de881c87c31504b8ae"
server_key_version: "encryptionkey@0"
I20260806 05:00:50.551939 32294 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785992227050689-9601-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785992227050689-9601-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785992227050689-9601-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 05:00:50.569036 32294 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 05:00:50.572992 32294 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 05:00:50.573341 32294 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 05:00:50.574399 32294 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 05:00:50.576277 32384 ts_tablet_manager.cc:542] Loading tablet metadata (0/1 complete)
I20260806 05:00:50.582062 32294 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260806 05:00:50.582217 32294 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.006s user 0.001s sys 0.000s
I20260806 05:00:50.582377 32294 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260806 05:00:50.585552 32294 ts_tablet_manager.cc:610] Registered 1 tablets
I20260806 05:00:50.585664 32294 ts_tablet_manager.cc:589] Time spent register tablets: real 0.003s user 0.001s sys 0.000s
I20260806 05:00:50.585991 32384 tablet_bootstrap.cc:492] T 51bd777d6f1f48a2a126fabf107583ae P 7bf95bbc6b8240e4b043b56c9cf673dc: Bootstrap starting.
I20260806 05:00:50.595196 32384 log.cc:824] T 51bd777d6f1f48a2a126fabf107583ae P 7bf95bbc6b8240e4b043b56c9cf673dc: Log is configured to *not* fsync() on all Append() calls
I20260806 05:00:50.610859 32384 tablet_bootstrap.cc:492] T 51bd777d6f1f48a2a126fabf107583ae P 7bf95bbc6b8240e4b043b56c9cf673dc: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 05:00:50.611658 32384 tablet_bootstrap.cc:492] T 51bd777d6f1f48a2a126fabf107583ae P 7bf95bbc6b8240e4b043b56c9cf673dc: Bootstrap complete.
I20260806 05:00:50.612156 32384 ts_tablet_manager.cc:1397] T 51bd777d6f1f48a2a126fabf107583ae P 7bf95bbc6b8240e4b043b56c9cf673dc: Time spent bootstrapping tablet: real 0.026s user 0.021s sys 0.004s
I20260806 05:00:50.621196 32384 raft_consensus.cc:348] T 51bd777d6f1f48a2a126fabf107583ae P 7bf95bbc6b8240e4b043b56c9cf673dc [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e35e3efb1eb4b289f33931312c99bfb" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 41983 } } peers { permanent_uuid: "2f286e3a16b847239e12defeff122bec" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 43613 } } peers { permanent_uuid: "7bf95bbc6b8240e4b043b56c9cf673dc" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 38017 } }
I20260806 05:00:50.622382 32384 raft_consensus.cc:729] T 51bd777d6f1f48a2a126fabf107583ae P 7bf95bbc6b8240e4b043b56c9cf673dc [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7bf95bbc6b8240e4b043b56c9cf673dc, State: Initialized, Role: FOLLOWER
I20260806 05:00:50.623261 32384 consensus_queue.cc:260] T 51bd777d6f1f48a2a126fabf107583ae P 7bf95bbc6b8240e4b043b56c9cf673dc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e35e3efb1eb4b289f33931312c99bfb" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 41983 } } peers { permanent_uuid: "2f286e3a16b847239e12defeff122bec" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 43613 } } peers { permanent_uuid: "7bf95bbc6b8240e4b043b56c9cf673dc" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 38017 } }
I20260806 05:00:50.628654 32384 ts_tablet_manager.cc:1428] T 51bd777d6f1f48a2a126fabf107583ae P 7bf95bbc6b8240e4b043b56c9cf673dc: Time spent starting tablet: real 0.016s user 0.012s sys 0.004s
I20260806 05:00:50.632112 32294 rpc_server.cc:307] RPC server started. Bound to: 127.9.96.65:38017
I20260806 05:00:50.632138 32544 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.96.65:38017 every 8 connection(s)
I20260806 05:00:50.633316 32294 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785992227050689-9601-0/minicluster-data/ts-0/data/info.pb
I20260806 05:00:50.637861 32549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:00:50.636024 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:39911 (local address 127.9.96.65:38447)
0806 05:00:50.636798 (+ 774us) negotiation.cc:107] Waiting for socket to connect
0806 05:00:50.637003 (+ 205us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.96.126:39911: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":510,"thread_start_us":316,"threads_started":1}
W20260806 05:00:50.638499 32546 heartbeater.cc:644] Failed to heartbeat to 127.9.96.126:39911 (0 consecutive failures): Network error: Failed to ping master at 127.9.96.126:39911: Client connection negotiation failed: client connection to 127.9.96.126:39911: connect: Connection refused (error 111)
I20260806 05:00:50.639317 32549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:00:50.638985 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:39911 (local address 127.9.96.65:54049)
0806 05:00:50.639137 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0806 05:00:50.639215 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.96.126:39911: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260806 05:00:50.640218 32549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:00:50.639867 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:39911 (local address 127.9.96.65:42171)
0806 05:00:50.640016 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0806 05:00:50.640108 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.96.126:39911: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260806 05:00:50.640287 9601 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu as pid 32294
W20260806 05:00:50.640543 32546 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260806 05:00:50.641011 9601 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
/tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785992227050689-9601-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785992227050689-9601-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785992227050689-9601-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785992227050689-9601-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.96.66:43613
--local_ip_for_outbound_sockets=127.9.96.66
--tserver_master_addrs=127.9.96.126:39911
--webserver_port=45935
--webserver_interface=127.9.96.66
--keytab_file=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.66.keytab
--principal=kudu/127.9.96.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.96.84:40741
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785992227050689-9601-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785992227050689-9601-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785992227050689-9601-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 05:00:50.979302 32550 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 05:00:50.979614 32550 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 05:00:50.979833 32550 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 05:00:50.987092 32550 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 05:00:50.987202 32550 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 05:00:50.987427 32550 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.96.66
I20260806 05:00:50.997977 32550 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.96.84:40741
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785992227050689-9601-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785992227050689-9601-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.66.keytab
--principal=kudu/127.9.96.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.96.66:43613
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785992227050689-9601-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.96.66
--webserver_port=45935
--tserver_master_addrs=127.9.96.126:39911
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.96.66
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785992227050689-9601-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 03:56:12 UTC on ccycloud.quasar-guijbm.root.comops.site
build id 4569618
ASAN enabled
I20260806 05:00:50.999244 32550 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 05:00:51.000602 32550 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 05:00:51.013995 32559 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 05:00:51.014083 32561 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 05:00:51.014083 32556 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 05:00:51.015044 32550 server_base.cc:1034] running on GCE node
I20260806 05:00:51.015690 32550 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 05:00:51.016593 32550 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 05:00:51.017788 32550 hybrid_clock.cc:648] HybridClock initialized: now 1785992451017736 us; error 48 us; skew 500 ppm
I20260806 05:00:51.018179 32550 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 05:00:51 dist-test-slave-x4qh krb5kdc[30534](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992451, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.96.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 05:00:51.021406 32550 init.cc:377] Logged in from keytab as kudu/127.9.96.66@KRBTEST.COM (short username kudu)
I20260806 05:00:51.022982 32550 webserver.cc:460] Webserver started at http://127.9.96.66:45935/ using document root <none> and password file <none>
I20260806 05:00:51.023818 32550 fs_manager.cc:359] Metadata directory not provided
I20260806 05:00:51.023975 32550 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 05:00:51.029178 32550 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.002s
I20260806 05:00:51.031957 32574 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 05:00:51.033300 32550 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260806 05:00:51.033484 32550 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785992227050689-9601-0/minicluster-data/ts-1/data,/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785992227050689-9601-0/minicluster-data/ts-1/wal
uuid: "2f286e3a16b847239e12defeff122bec"
format_stamp: "Formatted at 2026-08-06 05:00:45 on dist-test-slave-x4qh"
server_key: "5b9483aab78a2362524a22fd7fa7400b"
server_key_iv: "d6f1132ffd9b9249b72b90a77a248723"
server_key_version: "encryptionkey@0"
I20260806 05:00:51.034605 32550 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785992227050689-9601-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785992227050689-9601-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785992227050689-9601-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 05:00:51.069366 32550 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 05:00:51.073796 32550 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 05:00:51.074091 32550 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 05:00:51.075126 32550 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 05:00:51.076498 32550 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 05:00:51.076634 32550 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 05:00:51.076767 32550 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 05:00:51.076872 32550 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 05:00:51.112481 32550 rpc_server.cc:307] RPC server started. Bound to: 127.9.96.66:43613
I20260806 05:00:51.112550 32755 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.96.66:43613 every 8 connection(s)
I20260806 05:00:51.113981 32550 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785992227050689-9601-0/minicluster-data/ts-1/data/info.pb
I20260806 05:00:51.118623 32761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:00:51.116874 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:39911 (local address 127.9.96.66:34235)
0806 05:00:51.117610 (+ 736us) negotiation.cc:107] Waiting for socket to connect
0806 05:00:51.117816 (+ 206us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.96.126:39911: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":493,"thread_start_us":290,"threads_started":1}
W20260806 05:00:51.119269 32757 heartbeater.cc:644] Failed to heartbeat to 127.9.96.126:39911 (0 consecutive failures): Network error: Failed to ping master at 127.9.96.126:39911: Client connection negotiation failed: client connection to 127.9.96.126:39911: connect: Connection refused (error 111)
I20260806 05:00:51.120116 9601 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu as pid 32550
I20260806 05:00:51.120164 32761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:00:51.119770 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:39911 (local address 127.9.96.66:60541)
0806 05:00:51.119950 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0806 05:00:51.120033 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.96.126:39911: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260806 05:00:51.120755 9601 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
/tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785992227050689-9601-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785992227050689-9601-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785992227050689-9601-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785992227050689-9601-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.96.67:41983
--local_ip_for_outbound_sockets=127.9.96.67
--tserver_master_addrs=127.9.96.126:39911
--webserver_port=37687
--webserver_interface=127.9.96.67
--keytab_file=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.67.keytab
--principal=kudu/127.9.96.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.96.84:40741
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785992227050689-9601-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785992227050689-9601-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785992227050689-9601-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 05:00:51.121136 32761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:00:51.120701 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:39911 (local address 127.9.96.66:55545)
0806 05:00:51.120881 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0806 05:00:51.120988 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.96.126:39911: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
W20260806 05:00:51.121454 32757 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260806 05:00:51.449606 32762 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 05:00:51.449908 32762 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 05:00:51.450076 32762 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 05:00:51.457244 32762 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 05:00:51.457388 32762 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 05:00:51.457561 32762 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.96.67
I20260806 05:00:51.466833 32762 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.96.84:40741
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785992227050689-9601-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785992227050689-9601-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.67.keytab
--principal=kudu/127.9.96.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.96.67:41983
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785992227050689-9601-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.96.67
--webserver_port=37687
--tserver_master_addrs=127.9.96.126:39911
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.96.67
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785992227050689-9601-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 03:56:12 UTC on ccycloud.quasar-guijbm.root.comops.site
build id 4569618
ASAN enabled
I20260806 05:00:51.468120 32762 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 05:00:51.469400 32762 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 05:00:51.481845 302 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 05:00:51.481886 303 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 05:00:51.482159 307 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 05:00:51.482976 32762 server_base.cc:1034] running on GCE node
I20260806 05:00:51.483588 32762 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 05:00:51.484421 32762 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 05:00:51.485623 32762 hybrid_clock.cc:648] HybridClock initialized: now 1785992451485580 us; error 40 us; skew 500 ppm
I20260806 05:00:51.485939 32762 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 05:00:51 dist-test-slave-x4qh krb5kdc[30534](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992451, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.96.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 05:00:51.489037 32762 init.cc:377] Logged in from keytab as kudu/127.9.96.67@KRBTEST.COM (short username kudu)
I20260806 05:00:51.490355 32762 webserver.cc:460] Webserver started at http://127.9.96.67:37687/ using document root <none> and password file <none>
I20260806 05:00:51.490960 32762 fs_manager.cc:359] Metadata directory not provided
I20260806 05:00:51.491056 32762 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 05:00:51.496099 32762 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260806 05:00:51.498643 318 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 05:00:51.499935 32762 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260806 05:00:51.500129 32762 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785992227050689-9601-0/minicluster-data/ts-2/data,/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785992227050689-9601-0/minicluster-data/ts-2/wal
uuid: "9e35e3efb1eb4b289f33931312c99bfb"
format_stamp: "Formatted at 2026-08-06 05:00:45 on dist-test-slave-x4qh"
server_key: "12f5b45d70c62786077f37941913201a"
server_key_iv: "a9a117ada6701acebc9a8fac6fecd23e"
server_key_version: "encryptionkey@0"
I20260806 05:00:51.501077 32762 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785992227050689-9601-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785992227050689-9601-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785992227050689-9601-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 05:00:51.517751 32762 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 05:00:51.521549 32762 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 05:00:51.521826 32762 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 05:00:51.522710 32762 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 05:00:51.524003 32762 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 05:00:51.524098 32762 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 05:00:51.524168 32762 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 05:00:51.524216 32762 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 05:00:51.557646 32762 rpc_server.cc:307] RPC server started. Bound to: 127.9.96.67:41983
I20260806 05:00:51.557698 487 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.96.67:41983 every 8 connection(s)
I20260806 05:00:51.559111 32762 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785992227050689-9601-0/minicluster-data/ts-2/data/info.pb
I20260806 05:00:51.563865 495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:00:51.562053 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:39911 (local address 127.9.96.67:53951)
0806 05:00:51.562757 (+ 704us) negotiation.cc:107] Waiting for socket to connect
0806 05:00:51.562960 (+ 203us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.96.126:39911: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":467,"thread_start_us":269,"threads_started":1}
W20260806 05:00:51.564584 488 heartbeater.cc:644] Failed to heartbeat to 127.9.96.126:39911 (0 consecutive failures): Network error: Failed to ping master at 127.9.96.126:39911: Client connection negotiation failed: client connection to 127.9.96.126:39911: connect: Connection refused (error 111)
I20260806 05:00:51.565346 495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:00:51.565048 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:39911 (local address 127.9.96.67:53861)
0806 05:00:51.565178 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0806 05:00:51.565242 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.96.126:39911: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260806 05:00:51.566128 495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:00:51.565815 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:39911 (local address 127.9.96.67:55639)
0806 05:00:51.565954 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0806 05:00:51.566032 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.96.126:39911: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260806 05:00:51.566370 488 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260806 05:00:51.568109 9601 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu as pid 32762
I20260806 05:00:51.641762 497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:00:51.641004 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:39911 (local address 127.9.96.65:41765)
0806 05:00:51.641496 (+ 492us) negotiation.cc:107] Waiting for socket to connect
0806 05:00:51.641576 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.96.126:39911: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":392,"thread_start_us":267,"threads_started":1}
I20260806 05:00:52.122763 498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:00:52.122033 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:39911 (local address 127.9.96.66:43265)
0806 05:00:52.122468 (+ 435us) negotiation.cc:107] Waiting for socket to connect
0806 05:00:52.122548 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.96.126:39911: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":339,"thread_start_us":200,"threads_started":1}
I20260806 05:00:52.458447 502 raft_consensus.cc:482] T 51bd777d6f1f48a2a126fabf107583ae P 7bf95bbc6b8240e4b043b56c9cf673dc [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 05:00:52.458881 502 raft_consensus.cc:504] T 51bd777d6f1f48a2a126fabf107583ae P 7bf95bbc6b8240e4b043b56c9cf673dc [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e35e3efb1eb4b289f33931312c99bfb" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 41983 } } peers { permanent_uuid: "2f286e3a16b847239e12defeff122bec" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 43613 } } peers { permanent_uuid: "7bf95bbc6b8240e4b043b56c9cf673dc" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 38017 } }
I20260806 05:00:52.460702 502 leader_election.cc:288] T 51bd777d6f1f48a2a126fabf107583ae P 7bf95bbc6b8240e4b043b56c9cf673dc [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9e35e3efb1eb4b289f33931312c99bfb (127.9.96.67:41983), 2f286e3a16b847239e12defeff122bec (127.9.96.66:43613)
Aug 06 05:00:52 dist-test-slave-x4qh krb5kdc[30534](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992450, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.96.65@KRBTEST.COM for kudu/127.9.96.66@KRBTEST.COM
Aug 06 05:00:52 dist-test-slave-x4qh krb5kdc[30534](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992450, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.96.65@KRBTEST.COM for kudu/127.9.96.67@KRBTEST.COM
I20260806 05:00:52.500535 507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:00:52.461202 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.96.65:45639 (local address 127.9.96.67:41983)
0806 05:00:52.473055 (+ 11853us) server_negotiation.cc:206] Beginning negotiation
0806 05:00:52.473081 (+ 26us) server_negotiation.cc:399] Waiting for connection header
0806 05:00:52.473828 (+ 747us) server_negotiation.cc:407] Connection header received
0806 05:00:52.475969 (+ 2141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 05:00:52.475978 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 05:00:52.476330 (+ 352us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 05:00:52.476670 (+ 340us) server_negotiation.cc:226] Negotiated authn=SASL
0806 05:00:52.478737 (+ 2067us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:00:52.479979 (+ 1242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:00:52.482624 (+ 2645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:00:52.483670 (+ 1046us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:00:52.483857 (+ 187us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:00:52.493632 (+ 9775us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 05:00:52.493639 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 05:00:52.493649 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 05:00:52.493686 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0806 05:00:52.497092 (+ 3406us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 05:00:52.497880 (+ 788us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 05:00:52.497890 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 05:00:52.497898 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0806 05:00:52.498070 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 05:00:52.498475 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 05:00:52.498478 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 05:00:52.498480 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 05:00:52.499076 (+ 596us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 05:00:52.499253 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0806 05:00:52.499745 (+ 492us) server_negotiation.cc:299] Negotiation successful
0806 05:00:52.500111 (+ 366us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11553,"thread_start_us":11406,"threads_started":1}
I20260806 05:00:52.500530 508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:00:52.461520 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.96.65:47253 (local address 127.9.96.66:43613)
0806 05:00:52.473878 (+ 12358us) server_negotiation.cc:206] Beginning negotiation
0806 05:00:52.473893 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 05:00:52.474441 (+ 548us) server_negotiation.cc:407] Connection header received
0806 05:00:52.476116 (+ 1675us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 05:00:52.476124 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 05:00:52.476458 (+ 334us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 05:00:52.476766 (+ 308us) server_negotiation.cc:226] Negotiated authn=SASL
0806 05:00:52.478743 (+ 1977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:00:52.479981 (+ 1238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:00:52.482624 (+ 2643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:00:52.483670 (+ 1046us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:00:52.483860 (+ 190us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:00:52.493379 (+ 9519us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 05:00:52.493392 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 05:00:52.493409 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 05:00:52.493469 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0806 05:00:52.496409 (+ 2940us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 05:00:52.497267 (+ 858us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 05:00:52.497275 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 05:00:52.497283 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0806 05:00:52.497478 (+ 195us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 05:00:52.497952 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 05:00:52.497955 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 05:00:52.497957 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 05:00:52.498607 (+ 650us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 05:00:52.498738 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0806 05:00:52.499720 (+ 982us) server_negotiation.cc:299] Negotiation successful
0806 05:00:52.500114 (+ 394us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12186,"thread_start_us":12019,"threads_started":1}
I20260806 05:00:52.500550 504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:00:52.460922 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.67:41983 (local address 127.9.96.65:45639)
0806 05:00:52.473396 (+ 12474us) negotiation.cc:107] Waiting for socket to connect
0806 05:00:52.473455 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0806 05:00:52.474167 (+ 712us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 05:00:52.476778 (+ 2611us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 05:00:52.476790 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 05:00:52.477783 (+ 993us) client_negotiation.cc:189] Negotiated authn=SASL
0806 05:00:52.478545 (+ 762us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 05:00:52.478564 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 05:00:52.480174 (+ 1610us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 05:00:52.480179 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 05:00:52.482458 (+ 2279us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 05:00:52.482473 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 05:00:52.483909 (+ 1436us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 05:00:52.483916 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 05:00:52.484259 (+ 343us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:00:52.489668 (+ 5409us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 05:00:52.489713 (+ 45us) client_negotiation.cc:656] calling sasl_client_start()
0806 05:00:52.493386 (+ 3673us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 05:00:52.497260 (+ 3874us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 05:00:52.497265 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 05:00:52.497268 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 05:00:52.497745 (+ 477us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 05:00:52.498203 (+ 458us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 05:00:52.498210 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 05:00:52.498213 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 05:00:52.498371 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 05:00:52.499213 (+ 842us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 05:00:52.499216 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 05:00:52.499331 (+ 115us) client_negotiation.cc:769] Sending connection context
0806 05:00:52.499513 (+ 182us) client_negotiation.cc:240] Negotiation successful
0806 05:00:52.500098 (+ 585us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12278,"mutex_wait_us":353,"spinlock_wait_cycles":24064,"thread_start_us":12203,"threads_started":1}
I20260806 05:00:52.501032 505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:00:52.461155 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.66:43613 (local address 127.9.96.65:47253)
0806 05:00:52.474288 (+ 13133us) negotiation.cc:107] Waiting for socket to connect
0806 05:00:52.474303 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 05:00:52.474455 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 05:00:52.476779 (+ 2324us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 05:00:52.476789 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 05:00:52.477782 (+ 993us) client_negotiation.cc:189] Negotiated authn=SASL
0806 05:00:52.478545 (+ 763us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 05:00:52.478565 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 05:00:52.480178 (+ 1613us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 05:00:52.480182 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 05:00:52.482458 (+ 2276us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 05:00:52.482473 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 05:00:52.483923 (+ 1450us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 05:00:52.483929 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 05:00:52.484258 (+ 329us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:00:52.489668 (+ 5410us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 05:00:52.489713 (+ 45us) client_negotiation.cc:656] calling sasl_client_start()
0806 05:00:52.493048 (+ 3335us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 05:00:52.496571 (+ 3523us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 05:00:52.496580 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 05:00:52.496594 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0806 05:00:52.497142 (+ 548us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 05:00:52.497595 (+ 453us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 05:00:52.497598 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 05:00:52.497600 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 05:00:52.497834 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 05:00:52.498758 (+ 924us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 05:00:52.498765 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 05:00:52.499088 (+ 323us) client_negotiation.cc:769] Sending connection context
0806 05:00:52.499504 (+ 416us) client_negotiation.cc:240] Negotiation successful
0806 05:00:52.500275 (+ 771us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13036,"mutex_wait_us":558,"spinlock_wait_cycles":30080,"thread_start_us":12852,"threads_started":1}
W20260806 05:00:52.504190 32378 leader_election.cc:341] T 51bd777d6f1f48a2a126fabf107583ae P 7bf95bbc6b8240e4b043b56c9cf673dc [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 9e35e3efb1eb4b289f33931312c99bfb (127.9.96.67:41983): Not found: Tablet not found: 51bd777d6f1f48a2a126fabf107583ae
W20260806 05:00:52.504473 32381 leader_election.cc:341] T 51bd777d6f1f48a2a126fabf107583ae P 7bf95bbc6b8240e4b043b56c9cf673dc [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 2f286e3a16b847239e12defeff122bec (127.9.96.66:43613): Not found: Tablet not found: 51bd777d6f1f48a2a126fabf107583ae
I20260806 05:00:52.504618 32381 leader_election.cc:302] T 51bd777d6f1f48a2a126fabf107583ae P 7bf95bbc6b8240e4b043b56c9cf673dc [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bf95bbc6b8240e4b043b56c9cf673dc; no voters: 2f286e3a16b847239e12defeff122bec, 9e35e3efb1eb4b289f33931312c99bfb
I20260806 05:00:52.505077 502 raft_consensus.cc:2726] T 51bd777d6f1f48a2a126fabf107583ae P 7bf95bbc6b8240e4b043b56c9cf673dc [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 05:00:52.567680 510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:00:52.566828 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:39911 (local address 127.9.96.67:38187)
0806 05:00:52.567266 (+ 438us) negotiation.cc:107] Waiting for socket to connect
0806 05:00:52.567443 (+ 177us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.96.126:39911: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":347,"thread_start_us":220,"threads_started":1}
I20260806 05:00:52.643041 505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:00:52.642602 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:39911 (local address 127.9.96.65:51019)
0806 05:00:52.642771 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0806 05:00:52.642884 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.96.126:39911: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
I20260806 05:00:53.124432 511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:00:53.123768 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:39911 (local address 127.9.96.66:39149)
0806 05:00:53.124187 (+ 419us) negotiation.cc:107] Waiting for socket to connect
0806 05:00:53.124260 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.96.126:39911: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":329,"thread_start_us":234,"threads_started":1}
I20260806 05:00:53.569188 512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:00:53.568503 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:39911 (local address 127.9.96.67:34627)
0806 05:00:53.568965 (+ 462us) negotiation.cc:107] Waiting for socket to connect
0806 05:00:53.569042 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.96.126:39911: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":371,"thread_start_us":264,"threads_started":1}
I20260806 05:00:53.644562 513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:00:53.643873 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:39911 (local address 127.9.96.65:47423)
0806 05:00:53.644292 (+ 419us) negotiation.cc:107] Waiting for socket to connect
0806 05:00:53.644393 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.96.126:39911: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":328,"thread_start_us":214,"threads_started":1}
I20260806 05:00:54.125882 515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:00:54.125220 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:39911 (local address 127.9.96.66:41855)
0806 05:00:54.125623 (+ 403us) negotiation.cc:107] Waiting for socket to connect
0806 05:00:54.125696 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.96.126:39911: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":315,"thread_start_us":215,"threads_started":1}
I20260806 05:00:54.401674 517 raft_consensus.cc:482] T 51bd777d6f1f48a2a126fabf107583ae P 7bf95bbc6b8240e4b043b56c9cf673dc [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 05:00:54.401885 517 raft_consensus.cc:504] T 51bd777d6f1f48a2a126fabf107583ae P 7bf95bbc6b8240e4b043b56c9cf673dc [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e35e3efb1eb4b289f33931312c99bfb" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 41983 } } peers { permanent_uuid: "2f286e3a16b847239e12defeff122bec" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 43613 } } peers { permanent_uuid: "7bf95bbc6b8240e4b043b56c9cf673dc" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 38017 } }
I20260806 05:00:54.402580 517 leader_election.cc:288] T 51bd777d6f1f48a2a126fabf107583ae P 7bf95bbc6b8240e4b043b56c9cf673dc [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9e35e3efb1eb4b289f33931312c99bfb (127.9.96.67:41983), 2f286e3a16b847239e12defeff122bec (127.9.96.66:43613)
W20260806 05:00:54.403576 32378 leader_election.cc:341] T 51bd777d6f1f48a2a126fabf107583ae P 7bf95bbc6b8240e4b043b56c9cf673dc [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 9e35e3efb1eb4b289f33931312c99bfb (127.9.96.67:41983): Not found: Tablet not found: 51bd777d6f1f48a2a126fabf107583ae
W20260806 05:00:54.403817 32381 leader_election.cc:341] T 51bd777d6f1f48a2a126fabf107583ae P 7bf95bbc6b8240e4b043b56c9cf673dc [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 2f286e3a16b847239e12defeff122bec (127.9.96.66:43613): Not found: Tablet not found: 51bd777d6f1f48a2a126fabf107583ae
I20260806 05:00:54.403956 32381 leader_election.cc:302] T 51bd777d6f1f48a2a126fabf107583ae P 7bf95bbc6b8240e4b043b56c9cf673dc [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bf95bbc6b8240e4b043b56c9cf673dc; no voters: 2f286e3a16b847239e12defeff122bec, 9e35e3efb1eb4b289f33931312c99bfb
I20260806 05:00:54.404222 517 raft_consensus.cc:2726] T 51bd777d6f1f48a2a126fabf107583ae P 7bf95bbc6b8240e4b043b56c9cf673dc [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 05:00:54.570778 518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:00:54.570008 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:39911 (local address 127.9.96.67:55257)
0806 05:00:54.570551 (+ 543us) negotiation.cc:107] Waiting for socket to connect
0806 05:00:54.570628 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.96.126:39911: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":444,"thread_start_us":321,"threads_started":1}
I20260806 05:00:54.646132 520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:00:54.645377 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:39911 (local address 127.9.96.65:46077)
0806 05:00:54.645845 (+ 468us) negotiation.cc:107] Waiting for socket to connect
0806 05:00:54.645939 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.96.126:39911: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":349,"thread_start_us":218,"threads_started":1}
I20260806 05:00:55.127346 521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:00:55.126656 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:39911 (local address 127.9.96.66:35715)
0806 05:00:55.127090 (+ 434us) negotiation.cc:107] Waiting for socket to connect
0806 05:00:55.127186 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.96.126:39911: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":342,"thread_start_us":228,"threads_started":1}
I20260806 05:00:55.572396 522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:00:55.571710 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:39911 (local address 127.9.96.67:38203)
0806 05:00:55.572136 (+ 426us) negotiation.cc:107] Waiting for socket to connect
0806 05:00:55.572212 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.96.126:39911: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":330,"thread_start_us":218,"threads_started":1}
I20260806 05:00:55.647862 523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:00:55.647070 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:39911 (local address 127.9.96.65:44523)
0806 05:00:55.647596 (+ 526us) negotiation.cc:107] Waiting for socket to connect
0806 05:00:55.647673 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.96.126:39911: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":439,"thread_start_us":294,"threads_started":1}
I20260806 05:00:56.128907 524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:00:56.128254 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:39911 (local address 127.9.96.66:52821)
0806 05:00:56.128672 (+ 418us) negotiation.cc:107] Waiting for socket to connect
0806 05:00:56.128756 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.96.126:39911: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":332,"thread_start_us":214,"threads_started":1}
I20260806 05:00:56.573887 525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:00:56.573217 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:39911 (local address 127.9.96.67:47665)
0806 05:00:56.573654 (+ 437us) negotiation.cc:107] Waiting for socket to connect
0806 05:00:56.573727 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.96.126:39911: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":339,"thread_start_us":236,"threads_started":1}
I20260806 05:00:56.649295 526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:00:56.648616 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:39911 (local address 127.9.96.65:41489)
0806 05:00:56.649047 (+ 431us) negotiation.cc:107] Waiting for socket to connect
0806 05:00:56.649142 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.96.126:39911: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":340,"thread_start_us":230,"threads_started":1}
I20260806 05:00:57.130548 527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:00:57.129816 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:39911 (local address 127.9.96.66:60521)
0806 05:00:57.130280 (+ 464us) negotiation.cc:107] Waiting for socket to connect
0806 05:00:57.130362 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.96.126:39911: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":352,"thread_start_us":235,"threads_started":1}
I20260806 05:00:57.575467 528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:00:57.574688 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:39911 (local address 127.9.96.67:57141)
0806 05:00:57.575153 (+ 465us) negotiation.cc:107] Waiting for socket to connect
0806 05:00:57.575230 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.96.126:39911: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":374,"thread_start_us":258,"threads_started":1}
I20260806 05:00:57.650789 529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:00:57.650102 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:39911 (local address 127.9.96.65:55997)
0806 05:00:57.650530 (+ 428us) negotiation.cc:107] Waiting for socket to connect
0806 05:00:57.650603 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.96.126:39911: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":342,"thread_start_us":248,"threads_started":1}
I20260806 05:00:57.695672 530 raft_consensus.cc:482] T 51bd777d6f1f48a2a126fabf107583ae P 7bf95bbc6b8240e4b043b56c9cf673dc [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 05:00:57.695848 530 raft_consensus.cc:504] T 51bd777d6f1f48a2a126fabf107583ae P 7bf95bbc6b8240e4b043b56c9cf673dc [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e35e3efb1eb4b289f33931312c99bfb" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 41983 } } peers { permanent_uuid: "2f286e3a16b847239e12defeff122bec" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 43613 } } peers { permanent_uuid: "7bf95bbc6b8240e4b043b56c9cf673dc" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 38017 } }
I20260806 05:00:57.696449 530 leader_election.cc:288] T 51bd777d6f1f48a2a126fabf107583ae P 7bf95bbc6b8240e4b043b56c9cf673dc [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9e35e3efb1eb4b289f33931312c99bfb (127.9.96.67:41983), 2f286e3a16b847239e12defeff122bec (127.9.96.66:43613)
W20260806 05:00:57.697429 32381 leader_election.cc:341] T 51bd777d6f1f48a2a126fabf107583ae P 7bf95bbc6b8240e4b043b56c9cf673dc [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 2f286e3a16b847239e12defeff122bec (127.9.96.66:43613): Not found: Tablet not found: 51bd777d6f1f48a2a126fabf107583ae
W20260806 05:00:57.697616 32378 leader_election.cc:341] T 51bd777d6f1f48a2a126fabf107583ae P 7bf95bbc6b8240e4b043b56c9cf673dc [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 9e35e3efb1eb4b289f33931312c99bfb (127.9.96.67:41983): Not found: Tablet not found: 51bd777d6f1f48a2a126fabf107583ae
I20260806 05:00:57.697717 32378 leader_election.cc:302] T 51bd777d6f1f48a2a126fabf107583ae P 7bf95bbc6b8240e4b043b56c9cf673dc [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bf95bbc6b8240e4b043b56c9cf673dc; no voters: 2f286e3a16b847239e12defeff122bec, 9e35e3efb1eb4b289f33931312c99bfb
I20260806 05:00:57.697947 530 raft_consensus.cc:2726] T 51bd777d6f1f48a2a126fabf107583ae P 7bf95bbc6b8240e4b043b56c9cf673dc [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 05:00:58.132541 533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:00:58.131554 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:39911 (local address 127.9.96.66:43071)
0806 05:00:58.132192 (+ 638us) negotiation.cc:107] Waiting for socket to connect
0806 05:00:58.132300 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.96.126:39911: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":489,"thread_start_us":282,"threads_started":1}
I20260806 05:00:58.577262 534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:00:58.576331 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:39911 (local address 127.9.96.67:33483)
0806 05:00:58.576888 (+ 557us) negotiation.cc:107] Waiting for socket to connect
0806 05:00:58.576984 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.96.126:39911: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":428,"thread_start_us":249,"threads_started":1}
I20260806 05:00:58.652446 536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:00:58.651719 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:39911 (local address 127.9.96.65:40469)
0806 05:00:58.652194 (+ 475us) negotiation.cc:107] Waiting for socket to connect
0806 05:00:58.652278 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.96.126:39911: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":375,"thread_start_us":261,"threads_started":1}
I20260806 05:00:59.134510 537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:00:59.133596 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:39911 (local address 127.9.96.66:58727)
0806 05:00:59.134162 (+ 566us) negotiation.cc:107] Waiting for socket to connect
0806 05:00:59.134265 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.96.126:39911: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":437,"thread_start_us":309,"threads_started":1}
I20260806 05:00:59.521347 540 raft_consensus.cc:482] T 51bd777d6f1f48a2a126fabf107583ae P 7bf95bbc6b8240e4b043b56c9cf673dc [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 05:00:59.521562 540 raft_consensus.cc:504] T 51bd777d6f1f48a2a126fabf107583ae P 7bf95bbc6b8240e4b043b56c9cf673dc [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e35e3efb1eb4b289f33931312c99bfb" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 41983 } } peers { permanent_uuid: "2f286e3a16b847239e12defeff122bec" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 43613 } } peers { permanent_uuid: "7bf95bbc6b8240e4b043b56c9cf673dc" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 38017 } }
I20260806 05:00:59.522235 540 leader_election.cc:288] T 51bd777d6f1f48a2a126fabf107583ae P 7bf95bbc6b8240e4b043b56c9cf673dc [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9e35e3efb1eb4b289f33931312c99bfb (127.9.96.67:41983), 2f286e3a16b847239e12defeff122bec (127.9.96.66:43613)
W20260806 05:00:59.523101 32381 leader_election.cc:341] T 51bd777d6f1f48a2a126fabf107583ae P 7bf95bbc6b8240e4b043b56c9cf673dc [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 2f286e3a16b847239e12defeff122bec (127.9.96.66:43613): Not found: Tablet not found: 51bd777d6f1f48a2a126fabf107583ae
W20260806 05:00:59.523305 32378 leader_election.cc:341] T 51bd777d6f1f48a2a126fabf107583ae P 7bf95bbc6b8240e4b043b56c9cf673dc [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 9e35e3efb1eb4b289f33931312c99bfb (127.9.96.67:41983): Not found: Tablet not found: 51bd777d6f1f48a2a126fabf107583ae
I20260806 05:00:59.523460 32378 leader_election.cc:302] T 51bd777d6f1f48a2a126fabf107583ae P 7bf95bbc6b8240e4b043b56c9cf673dc [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bf95bbc6b8240e4b043b56c9cf673dc; no voters: 2f286e3a16b847239e12defeff122bec, 9e35e3efb1eb4b289f33931312c99bfb
I20260806 05:00:59.523710 540 raft_consensus.cc:2726] T 51bd777d6f1f48a2a126fabf107583ae P 7bf95bbc6b8240e4b043b56c9cf673dc [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 05:00:59.578904 542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:00:59.578202 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:39911 (local address 127.9.96.67:53269)
0806 05:00:59.578642 (+ 440us) negotiation.cc:107] Waiting for socket to connect
0806 05:00:59.578721 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.96.126:39911: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":349,"thread_start_us":248,"threads_started":1}
I20260806 05:00:59.654081 543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:00:59.653273 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:39911 (local address 127.9.96.65:49995)
0806 05:00:59.653799 (+ 526us) negotiation.cc:107] Waiting for socket to connect
0806 05:00:59.653910 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.96.126:39911: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":429,"thread_start_us":277,"threads_started":1}
I20260806 05:01:00.136272 544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:01:00.135412 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:39911 (local address 127.9.96.66:53073)
0806 05:01:00.135987 (+ 575us) negotiation.cc:107] Waiting for socket to connect
0806 05:01:00.136085 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.96.126:39911: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":431,"thread_start_us":285,"threads_started":1}
I20260806 05:01:00.580502 547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:01:00.579784 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:39911 (local address 127.9.96.67:56723)
0806 05:01:00.580248 (+ 464us) negotiation.cc:107] Waiting for socket to connect
0806 05:01:00.580328 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.96.126:39911: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":361,"thread_start_us":222,"threads_started":1}
I20260806 05:01:00.655769 548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:01:00.655057 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:39911 (local address 127.9.96.65:35569)
0806 05:01:00.655530 (+ 473us) negotiation.cc:107] Waiting for socket to connect
0806 05:01:00.655619 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.96.126:39911: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":375,"thread_start_us":226,"threads_started":1}
I20260806 05:01:01.138154 549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:01:01.137252 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:39911 (local address 127.9.96.66:57413)
0806 05:01:01.137799 (+ 547us) negotiation.cc:107] Waiting for socket to connect
0806 05:01:01.137962 (+ 163us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.96.126:39911: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":427,"thread_start_us":302,"threads_started":1}
I20260806 05:01:01.582190 553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:01:01.581485 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:39911 (local address 127.9.96.67:60743)
0806 05:01:01.581932 (+ 447us) negotiation.cc:107] Waiting for socket to connect
0806 05:01:01.582011 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.96.126:39911: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":354,"thread_start_us":238,"threads_started":1}
I20260806 05:01:01.657452 554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:01:01.656714 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:39911 (local address 127.9.96.65:48853)
0806 05:01:01.657152 (+ 438us) negotiation.cc:107] Waiting for socket to connect
0806 05:01:01.657233 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.96.126:39911: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":338,"thread_start_us":233,"threads_started":1}
I20260806 05:01:01.963195 555 raft_consensus.cc:482] T 51bd777d6f1f48a2a126fabf107583ae P 7bf95bbc6b8240e4b043b56c9cf673dc [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 05:01:01.963454 555 raft_consensus.cc:504] T 51bd777d6f1f48a2a126fabf107583ae P 7bf95bbc6b8240e4b043b56c9cf673dc [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e35e3efb1eb4b289f33931312c99bfb" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 41983 } } peers { permanent_uuid: "2f286e3a16b847239e12defeff122bec" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 43613 } } peers { permanent_uuid: "7bf95bbc6b8240e4b043b56c9cf673dc" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 38017 } }
I20260806 05:01:01.964092 555 leader_election.cc:288] T 51bd777d6f1f48a2a126fabf107583ae P 7bf95bbc6b8240e4b043b56c9cf673dc [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9e35e3efb1eb4b289f33931312c99bfb (127.9.96.67:41983), 2f286e3a16b847239e12defeff122bec (127.9.96.66:43613)
W20260806 05:01:01.965134 32378 leader_election.cc:341] T 51bd777d6f1f48a2a126fabf107583ae P 7bf95bbc6b8240e4b043b56c9cf673dc [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 9e35e3efb1eb4b289f33931312c99bfb (127.9.96.67:41983): Not found: Tablet not found: 51bd777d6f1f48a2a126fabf107583ae
W20260806 05:01:01.965319 32381 leader_election.cc:341] T 51bd777d6f1f48a2a126fabf107583ae P 7bf95bbc6b8240e4b043b56c9cf673dc [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 2f286e3a16b847239e12defeff122bec (127.9.96.66:43613): Not found: Tablet not found: 51bd777d6f1f48a2a126fabf107583ae
I20260806 05:01:01.965420 32381 leader_election.cc:302] T 51bd777d6f1f48a2a126fabf107583ae P 7bf95bbc6b8240e4b043b56c9cf673dc [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bf95bbc6b8240e4b043b56c9cf673dc; no voters: 2f286e3a16b847239e12defeff122bec, 9e35e3efb1eb4b289f33931312c99bfb
I20260806 05:01:01.965636 555 raft_consensus.cc:2726] T 51bd777d6f1f48a2a126fabf107583ae P 7bf95bbc6b8240e4b043b56c9cf673dc [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 05:01:02.139825 556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:01:02.138991 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:39911 (local address 127.9.96.66:55261)
0806 05:01:02.139547 (+ 556us) negotiation.cc:107] Waiting for socket to connect
0806 05:01:02.139634 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.96.126:39911: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":446,"thread_start_us":291,"threads_started":1}
I20260806 05:01:02.584060 557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:01:02.583195 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:39911 (local address 127.9.96.67:44081)
0806 05:01:02.583813 (+ 618us) negotiation.cc:107] Waiting for socket to connect
0806 05:01:02.583906 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.96.126:39911: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":508,"thread_start_us":363,"threads_started":1}
I20260806 05:01:02.658967 558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:01:02.658277 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:39911 (local address 127.9.96.65:54121)
0806 05:01:02.658715 (+ 438us) negotiation.cc:107] Waiting for socket to connect
0806 05:01:02.658809 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.96.126:39911: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":349,"thread_start_us":246,"threads_started":1}
I20260806 05:01:03.141297 559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:01:03.140632 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:39911 (local address 127.9.96.66:39133)
0806 05:01:03.141046 (+ 414us) negotiation.cc:107] Waiting for socket to connect
0806 05:01:03.141125 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.96.126:39911: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":324,"thread_start_us":222,"threads_started":1}
I20260806 05:01:03.585762 561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:01:03.585012 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:39911 (local address 127.9.96.67:34131)
0806 05:01:03.585466 (+ 454us) negotiation.cc:107] Waiting for socket to connect
0806 05:01:03.585552 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.96.126:39911: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":359,"thread_start_us":247,"threads_started":1}
I20260806 05:01:03.660460 562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:01:03.659718 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:39911 (local address 127.9.96.65:42217)
0806 05:01:03.660211 (+ 493us) negotiation.cc:107] Waiting for socket to connect
0806 05:01:03.660286 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.96.126:39911: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":380,"thread_start_us":282,"threads_started":1}
I20260806 05:01:04.142829 564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:01:04.142143 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:39911 (local address 127.9.96.66:47469)
0806 05:01:04.142574 (+ 431us) negotiation.cc:107] Waiting for socket to connect
0806 05:01:04.142659 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.96.126:39911: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":342,"thread_start_us":232,"threads_started":1}
I20260806 05:01:04.587611 566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:01:04.586840 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:39911 (local address 127.9.96.67:60735)
0806 05:01:04.587298 (+ 458us) negotiation.cc:107] Waiting for socket to connect
0806 05:01:04.587453 (+ 155us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.96.126:39911: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":357,"thread_start_us":233,"threads_started":1}
I20260806 05:01:04.661995 572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:01:04.661262 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:39911 (local address 127.9.96.65:42643)
0806 05:01:04.661732 (+ 470us) negotiation.cc:107] Waiting for socket to connect
0806 05:01:04.661821 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.96.126:39911: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":380,"thread_start_us":244,"threads_started":1}
I20260806 05:01:05.144466 573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:01:05.143715 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:39911 (local address 127.9.96.66:32875)
0806 05:01:05.144166 (+ 451us) negotiation.cc:107] Waiting for socket to connect
0806 05:01:05.144257 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.96.126:39911: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":358,"thread_start_us":250,"threads_started":1}
I20260806 05:01:05.589380 574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:01:05.588527 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:39911 (local address 127.9.96.67:48879)
0806 05:01:05.589099 (+ 572us) negotiation.cc:107] Waiting for socket to connect
0806 05:01:05.589178 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.96.126:39911: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":472,"thread_start_us":320,"threads_started":1}
I20260806 05:01:05.664041 575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:01:05.663198 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:39911 (local address 127.9.96.65:49333)
0806 05:01:05.663757 (+ 559us) negotiation.cc:107] Waiting for socket to connect
0806 05:01:05.663857 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.96.126:39911: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":456,"thread_start_us":325,"threads_started":1}
I20260806 05:01:06.146289 576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:01:06.145502 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:39911 (local address 127.9.96.66:51619)
0806 05:01:06.145975 (+ 473us) negotiation.cc:107] Waiting for socket to connect
0806 05:01:06.146052 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.96.126:39911: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":352,"thread_start_us":235,"threads_started":1}
I20260806 05:01:06.572705 9601 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu pid 32294
I20260806 05:01:06.591284 578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:01:06.590554 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:39911 (local address 127.9.96.67:58171)
0806 05:01:06.591020 (+ 466us) negotiation.cc:107] Waiting for socket to connect
0806 05:01:06.591136 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.96.126:39911: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":369,"thread_start_us":265,"threads_started":1}
I20260806 05:01:06.593277 577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:01:06.573899 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41010 (local address 127.9.96.65:38017)
0806 05:01:06.574487 (+ 588us) server_negotiation.cc:206] Beginning negotiation
0806 05:01:06.574497 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 05:01:06.574609 (+ 112us) server_negotiation.cc:407] Connection header received
0806 05:01:06.574837 (+ 228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 05:01:06.574848 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 05:01:06.575185 (+ 337us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 05:01:06.575400 (+ 215us) server_negotiation.cc:226] Negotiated authn=SASL
0806 05:01:06.577222 (+ 1822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:01:06.578186 (+ 964us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:01:06.579343 (+ 1157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:01:06.579944 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:01:06.580038 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:01:06.587564 (+ 7526us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 05:01:06.587580 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 05:01:06.587595 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 05:01:06.587632 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0806 05:01:06.590610 (+ 2978us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 05:01:06.591423 (+ 813us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 05:01:06.591435 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 05:01:06.591438 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 05:01:06.591607 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 05:01:06.591980 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 05:01:06.591983 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 05:01:06.591985 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 05:01:06.592366 (+ 381us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 05:01:06.592573 (+ 207us) server_negotiation.cc:1035] Waiting for connection context
0806 05:01:06.592851 (+ 278us) server_negotiation.cc:299] Negotiation successful
0806 05:01:06.593041 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":417,"thread_start_us":253,"threads_started":1}
I20260806 05:01:06.594502 32537 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 05:01:06.682147 582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:01:06.681026 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:39911 (local address 127.9.96.65:47519)
0806 05:01:06.681650 (+ 624us) negotiation.cc:107] Waiting for socket to connect
0806 05:01:06.681814 (+ 164us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.96.126:39911: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":505,"thread_start_us":283,"threads_started":1}
I20260806 05:01:06.683566 9601 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu with pid 32294
I20260806 05:01:06.710891 9601 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu pid 32550
I20260806 05:01:06.724368 583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:01:06.711693 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55700 (local address 127.9.96.66:43613)
0806 05:01:06.712194 (+ 501us) server_negotiation.cc:206] Beginning negotiation
0806 05:01:06.712198 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 05:01:06.712213 (+ 15us) server_negotiation.cc:407] Connection header received
0806 05:01:06.712266 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 05:01:06.712270 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 05:01:06.712327 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 05:01:06.712443 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0806 05:01:06.713832 (+ 1389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:01:06.714548 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:01:06.715983 (+ 1435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:01:06.716426 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:01:06.716564 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:01:06.719248 (+ 2684us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 05:01:06.719253 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 05:01:06.719256 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 05:01:06.719288 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0806 05:01:06.721705 (+ 2417us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 05:01:06.722377 (+ 672us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 05:01:06.722382 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 05:01:06.722391 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0806 05:01:06.722526 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 05:01:06.722907 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 05:01:06.722912 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 05:01:06.722919 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 05:01:06.723187 (+ 268us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 05:01:06.723485 (+ 298us) server_negotiation.cc:1035] Waiting for connection context
0806 05:01:06.723754 (+ 269us) server_negotiation.cc:299] Negotiation successful
0806 05:01:06.723980 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":385,"thread_start_us":258,"threads_started":1}
I20260806 05:01:06.725147 32748 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 05:01:06.807762 9601 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu with pid 32550
I20260806 05:01:06.835183 9601 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu pid 32762
I20260806 05:01:06.852882 587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:01:06.836099 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37772 (local address 127.9.96.67:41983)
0806 05:01:06.836703 (+ 604us) server_negotiation.cc:206] Beginning negotiation
0806 05:01:06.836708 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 05:01:06.836723 (+ 15us) server_negotiation.cc:407] Connection header received
0806 05:01:06.836798 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 05:01:06.836802 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 05:01:06.836881 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 05:01:06.837007 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0806 05:01:06.838401 (+ 1394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:01:06.839210 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:01:06.840589 (+ 1379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:01:06.841039 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:01:06.841176 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:01:06.847521 (+ 6345us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 05:01:06.847526 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 05:01:06.847530 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 05:01:06.847563 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0806 05:01:06.850103 (+ 2540us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 05:01:06.850822 (+ 719us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 05:01:06.850828 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 05:01:06.850832 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 05:01:06.850995 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 05:01:06.851400 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 05:01:06.851420 (+ 20us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 05:01:06.851423 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 05:01:06.851692 (+ 269us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 05:01:06.851983 (+ 291us) server_negotiation.cc:1035] Waiting for connection context
0806 05:01:06.852245 (+ 262us) server_negotiation.cc:299] Negotiation successful
0806 05:01:06.852495 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":465,"thread_start_us":275,"threads_started":1}
I20260806 05:01:06.853608 480 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 05:01:06.936565 9601 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu with pid 32762
2026-08-06T05:01:06Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (24726 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785992227050689-9601-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 05:01:07 dist-test-slave-x4qh krb5kdc[593](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 05:01:07 dist-test-slave-x4qh krb5kdc[593](info): set up 2 sockets
Aug 06 05:01:07 dist-test-slave-x4qh krb5kdc[593](info): commencing operation
krb5kdc: starting...
W20260806 05:01:08.261940 9601 mini_kdc.cc:121] Time spent starting KDC: real 1.275s user 0.004s sys 0.033s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785992227050689-9601-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785992227050689-9601-0/krb5kdc/test-user.keytab.
Aug 06 05:01:08 dist-test-slave-x4qh krb5kdc[593](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992468, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T05:01:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T05:01:08Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.96.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.96.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.96.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.126.keytab.
Entry for principal kudu/127.9.96.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.126.keytab.
I20260806 05:01:08.526401 9601 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
/tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785992227050689-9601-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785992227050689-9601-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785992227050689-9601-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785992227050689-9601-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.96.126:34193
--webserver_interface=127.9.96.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.126.keytab
--principal=kudu/127.9.96.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.96.84:41431
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.96.126:34193
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785992227050689-9601-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785992227050689-9601-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785992227050689-9601-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 05:01:08.853998 615 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 05:01:08.854367 615 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 05:01:08.854542 615 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 05:01:08.861574 615 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260806 05:01:08.861716 615 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260806 05:01:08.861809 615 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260806 05:01:08.861891 615 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 05:01:08.861994 615 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 05:01:08.862102 615 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 05:01:08.862191 615 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 05:01:08.862275 615 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 05:01:08.871755 615 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.96.84:41431
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785992227050689-9601-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785992227050689-9601-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.9.96.126:34193
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.126.keytab
--principal=kudu/127.9.96.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.96.126:34193
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785992227050689-9601-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.96.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785992227050689-9601-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 03:56:12 UTC on ccycloud.quasar-guijbm.root.comops.site
build id 4569618
ASAN enabled
I20260806 05:01:08.872988 615 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 05:01:08.874320 615 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 05:01:08.886710 625 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 05:01:08.886960 627 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 05:01:08.886744 624 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 05:01:08.887658 615 server_base.cc:1034] running on GCE node
I20260806 05:01:08.888538 615 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 05:01:08.889912 615 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 05:01:08.891165 615 hybrid_clock.cc:648] HybridClock initialized: now 1785992468891112 us; error 56 us; skew 500 ppm
I20260806 05:01:08.891556 615 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 05:01:08 dist-test-slave-x4qh krb5kdc[593](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992468, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.96.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 05:01:08.894685 615 init.cc:377] Logged in from keytab as kudu/127.9.96.126@KRBTEST.COM (short username kudu)
I20260806 05:01:08.896168 615 webserver.cc:460] Webserver started at http://127.9.96.126:33641/ using document root <none> and password file <none>
I20260806 05:01:08.896857 615 fs_manager.cc:359] Metadata directory not provided
I20260806 05:01:08.896992 615 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 05:01:08.897318 615 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 05:01:08.900444 615 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785992227050689-9601-0/minicluster-data/master-0/data/instance:
uuid: "28af3f4cbb8141709b39138ff8f369b3"
format_stamp: "Formatted at 2026-08-06 05:01:08 on dist-test-slave-x4qh"
server_key: "c36b42d5d25059f8032ec59f36e1c6cf"
server_key_iv: "97e75fb038a44ae429b93be19508ec27"
server_key_version: "encryptionkey@0"
I20260806 05:01:08.901227 615 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785992227050689-9601-0/minicluster-data/master-0/wal/instance:
uuid: "28af3f4cbb8141709b39138ff8f369b3"
format_stamp: "Formatted at 2026-08-06 05:01:08 on dist-test-slave-x4qh"
server_key: "c36b42d5d25059f8032ec59f36e1c6cf"
server_key_iv: "97e75fb038a44ae429b93be19508ec27"
server_key_version: "encryptionkey@0"
I20260806 05:01:08.906178 615 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260806 05:01:08.909587 645 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 05:01:08.910776 615 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.001s
I20260806 05:01:08.910955 615 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785992227050689-9601-0/minicluster-data/master-0/data,/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785992227050689-9601-0/minicluster-data/master-0/wal
uuid: "28af3f4cbb8141709b39138ff8f369b3"
format_stamp: "Formatted at 2026-08-06 05:01:08 on dist-test-slave-x4qh"
server_key: "c36b42d5d25059f8032ec59f36e1c6cf"
server_key_iv: "97e75fb038a44ae429b93be19508ec27"
server_key_version: "encryptionkey@0"
I20260806 05:01:08.911172 615 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785992227050689-9601-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785992227050689-9601-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785992227050689-9601-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 05:01:08.955225 615 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 05:01:08.959587 615 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 05:01:08.959934 615 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 05:01:08.979709 615 rpc_server.cc:307] RPC server started. Bound to: 127.9.96.126:34193
I20260806 05:01:08.979745 720 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.96.126:34193 every 8 connection(s)
I20260806 05:01:08.981093 615 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785992227050689-9601-0/minicluster-data/master-0/data/info.pb
I20260806 05:01:08.984859 721 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 05:01:08.990134 9601 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu as pid 615
I20260806 05:01:08.990383 9601 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785992227050689-9601-0/minicluster-data/master-0/wal/instance
I20260806 05:01:08.991039 9601 external_mini_cluster.cc:1428] Setting key e94168fff87a73d22904efb51ccbece5
I20260806 05:01:08.995793 721 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 28af3f4cbb8141709b39138ff8f369b3: Bootstrap starting.
I20260806 05:01:08.999485 721 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 28af3f4cbb8141709b39138ff8f369b3: Neither blocks nor log segments found. Creating new log.
I20260806 05:01:09.000741 721 log.cc:824] T 00000000000000000000000000000000 P 28af3f4cbb8141709b39138ff8f369b3: Log is configured to *not* fsync() on all Append() calls
I20260806 05:01:09.003618 721 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 28af3f4cbb8141709b39138ff8f369b3: No bootstrap required, opened a new log
Aug 06 05:01:09 dist-test-slave-x4qh krb5kdc[593](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992468, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.96.126@KRBTEST.COM
I20260806 05:01:09.011477 721 raft_consensus.cc:348] T 00000000000000000000000000000000 P 28af3f4cbb8141709b39138ff8f369b3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "28af3f4cbb8141709b39138ff8f369b3" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 34193 } }
I20260806 05:01:09.011859 721 raft_consensus.cc:374] T 00000000000000000000000000000000 P 28af3f4cbb8141709b39138ff8f369b3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 05:01:09.011986 721 raft_consensus.cc:729] T 00000000000000000000000000000000 P 28af3f4cbb8141709b39138ff8f369b3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 28af3f4cbb8141709b39138ff8f369b3, State: Initialized, Role: FOLLOWER
I20260806 05:01:09.012789 721 consensus_queue.cc:260] T 00000000000000000000000000000000 P 28af3f4cbb8141709b39138ff8f369b3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "28af3f4cbb8141709b39138ff8f369b3" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 34193 } }
I20260806 05:01:09.013128 721 raft_consensus.cc:388] T 00000000000000000000000000000000 P 28af3f4cbb8141709b39138ff8f369b3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 05:01:09.013296 721 raft_consensus.cc:482] T 00000000000000000000000000000000 P 28af3f4cbb8141709b39138ff8f369b3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 05:01:09.013494 721 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 28af3f4cbb8141709b39138ff8f369b3 [term 0 FOLLOWER]: Advancing to term 1
I20260806 05:01:09.014673 727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:01:08.993110 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47886 (local address 127.9.96.126:34193)
0806 05:01:08.993984 (+ 874us) server_negotiation.cc:206] Beginning negotiation
0806 05:01:08.993995 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 05:01:08.994041 (+ 46us) server_negotiation.cc:407] Connection header received
0806 05:01:08.995748 (+ 1707us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 05:01:08.995770 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 05:01:08.996207 (+ 437us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 05:01:08.996514 (+ 307us) server_negotiation.cc:226] Negotiated authn=SASL
0806 05:01:08.998456 (+ 1942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:01:08.999888 (+ 1432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:01:09.001820 (+ 1932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:01:09.002471 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:01:09.002674 (+ 203us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:01:09.006821 (+ 4147us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 05:01:09.006829 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 05:01:09.006843 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 05:01:09.006884 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0806 05:01:09.010104 (+ 3220us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 05:01:09.010857 (+ 753us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 05:01:09.010873 (+ 16us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 05:01:09.010880 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 05:01:09.011049 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 05:01:09.011517 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 05:01:09.011523 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 05:01:09.011528 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 05:01:09.012184 (+ 656us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 05:01:09.012429 (+ 245us) server_negotiation.cc:1035] Waiting for connection context
0806 05:01:09.012913 (+ 484us) server_negotiation.cc:299] Negotiation successful
0806 05:01:09.013357 (+ 444us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":675,"thread_start_us":460,"threads_started":1}
I20260806 05:01:09.016323 721 raft_consensus.cc:504] T 00000000000000000000000000000000 P 28af3f4cbb8141709b39138ff8f369b3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "28af3f4cbb8141709b39138ff8f369b3" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 34193 } }
I20260806 05:01:09.016916 721 leader_election.cc:302] T 00000000000000000000000000000000 P 28af3f4cbb8141709b39138ff8f369b3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 28af3f4cbb8141709b39138ff8f369b3; no voters:
I20260806 05:01:09.017586 721 leader_election.cc:288] T 00000000000000000000000000000000 P 28af3f4cbb8141709b39138ff8f369b3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 05:01:09.017724 729 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 28af3f4cbb8141709b39138ff8f369b3 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 05:01:09.018411 729 raft_consensus.cc:686] T 00000000000000000000000000000000 P 28af3f4cbb8141709b39138ff8f369b3 [term 1 LEADER]: Becoming Leader. State: Replica: 28af3f4cbb8141709b39138ff8f369b3, State: Running, Role: LEADER
I20260806 05:01:09.019068 729 consensus_queue.cc:237] T 00000000000000000000000000000000 P 28af3f4cbb8141709b39138ff8f369b3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "28af3f4cbb8141709b39138ff8f369b3" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 34193 } }
I20260806 05:01:09.020077 721 sys_catalog.cc:564] T 00000000000000000000000000000000 P 28af3f4cbb8141709b39138ff8f369b3 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 05:01:09.025372 730 sys_catalog.cc:455] T 00000000000000000000000000000000 P 28af3f4cbb8141709b39138ff8f369b3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 28af3f4cbb8141709b39138ff8f369b3. Latest consensus state: current_term: 1 leader_uuid: "28af3f4cbb8141709b39138ff8f369b3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "28af3f4cbb8141709b39138ff8f369b3" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 34193 } } }
I20260806 05:01:09.025812 730 sys_catalog.cc:458] T 00000000000000000000000000000000 P 28af3f4cbb8141709b39138ff8f369b3 [sys.catalog]: This master's current role is: LEADER
I20260806 05:01:09.025379 733 sys_catalog.cc:455] T 00000000000000000000000000000000 P 28af3f4cbb8141709b39138ff8f369b3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "28af3f4cbb8141709b39138ff8f369b3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "28af3f4cbb8141709b39138ff8f369b3" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 34193 } } }
I20260806 05:01:09.026542 733 sys_catalog.cc:458] T 00000000000000000000000000000000 P 28af3f4cbb8141709b39138ff8f369b3 [sys.catalog]: This master's current role is: LEADER
I20260806 05:01:09.028916 750 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 05:01:09.033205 750 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 05:01:09.042131 750 catalog_manager.cc:1347] Generated new cluster ID: d13035ae4ef14dfab1dc54b0a8fbf05e
I20260806 05:01:09.042250 750 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 05:01:09.070822 750 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 05:01:09.072193 750 catalog_manager.cc:1504] Loading token signing keys...
I20260806 05:01:09.081645 750 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 28af3f4cbb8141709b39138ff8f369b3: Generated new TSK 0
I20260806 05:01:09.082616 750 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.9.96.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.96.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.96.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.65.keytab.
Entry for principal kudu/127.9.96.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.65.keytab.
I20260806 05:01:09.154624 9601 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
/tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785992227050689-9601-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785992227050689-9601-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785992227050689-9601-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785992227050689-9601-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.96.65:0
--local_ip_for_outbound_sockets=127.9.96.65
--webserver_interface=127.9.96.65
--webserver_port=0
--tserver_master_addrs=127.9.96.126:34193
--keytab_file=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.65.keytab
--principal=kudu/127.9.96.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.96.84:41431
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785992227050689-9601-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785992227050689-9601-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785992227050689-9601-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 05:01:09.562105 766 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 05:01:09.562573 766 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 05:01:09.562758 766 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 05:01:09.570322 766 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 05:01:09.570461 766 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 05:01:09.570669 766 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.96.65
I20260806 05:01:09.580375 766 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.96.84:41431
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785992227050689-9601-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785992227050689-9601-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.65.keytab
--principal=kudu/127.9.96.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.96.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785992227050689-9601-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.96.65
--webserver_port=0
--tserver_master_addrs=127.9.96.126:34193
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.96.65
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785992227050689-9601-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 03:56:12 UTC on ccycloud.quasar-guijbm.root.comops.site
build id 4569618
ASAN enabled
I20260806 05:01:09.581857 766 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 05:01:09.583108 766 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 05:01:09.596451 771 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 05:01:09.596446 772 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 05:01:09.596778 775 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 05:01:09.597448 766 server_base.cc:1034] running on GCE node
I20260806 05:01:09.598021 766 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 05:01:09.598860 766 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 05:01:09.600059 766 hybrid_clock.cc:648] HybridClock initialized: now 1785992469600018 us; error 28 us; skew 500 ppm
I20260806 05:01:09.600379 766 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 05:01:09 dist-test-slave-x4qh krb5kdc[593](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992469, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.96.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 05:01:09.603614 766 init.cc:377] Logged in from keytab as kudu/127.9.96.65@KRBTEST.COM (short username kudu)
I20260806 05:01:09.605008 766 webserver.cc:460] Webserver started at http://127.9.96.65:44495/ using document root <none> and password file <none>
I20260806 05:01:09.605625 766 fs_manager.cc:359] Metadata directory not provided
I20260806 05:01:09.605726 766 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 05:01:09.605993 766 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 05:01:09.608846 766 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785992227050689-9601-0/minicluster-data/ts-0/data/instance:
uuid: "14be68f793794acaba3556a8e1ef3c13"
format_stamp: "Formatted at 2026-08-06 05:01:09 on dist-test-slave-x4qh"
server_key: "3128738e1e7ef5228a1daa36f098d2f4"
server_key_iv: "7922f313fa41aa73b12b8f5bc3e10018"
server_key_version: "encryptionkey@0"
I20260806 05:01:09.609546 766 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785992227050689-9601-0/minicluster-data/ts-0/wal/instance:
uuid: "14be68f793794acaba3556a8e1ef3c13"
format_stamp: "Formatted at 2026-08-06 05:01:09 on dist-test-slave-x4qh"
server_key: "3128738e1e7ef5228a1daa36f098d2f4"
server_key_iv: "7922f313fa41aa73b12b8f5bc3e10018"
server_key_version: "encryptionkey@0"
I20260806 05:01:09.614573 766 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260806 05:01:09.618103 785 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 05:01:09.619324 766 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260806 05:01:09.619509 766 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785992227050689-9601-0/minicluster-data/ts-0/data,/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785992227050689-9601-0/minicluster-data/ts-0/wal
uuid: "14be68f793794acaba3556a8e1ef3c13"
format_stamp: "Formatted at 2026-08-06 05:01:09 on dist-test-slave-x4qh"
server_key: "3128738e1e7ef5228a1daa36f098d2f4"
server_key_iv: "7922f313fa41aa73b12b8f5bc3e10018"
server_key_version: "encryptionkey@0"
I20260806 05:01:09.619670 766 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785992227050689-9601-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785992227050689-9601-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785992227050689-9601-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 05:01:09.637178 766 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 05:01:09.641194 766 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 05:01:09.641451 766 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 05:01:09.642307 766 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 05:01:09.643602 766 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 05:01:09.643676 766 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 05:01:09.643772 766 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 05:01:09.643824 766 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 05:01:09.678239 766 rpc_server.cc:307] RPC server started. Bound to: 127.9.96.65:45547
I20260806 05:01:09.678277 947 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.96.65:45547 every 8 connection(s)
I20260806 05:01:09.679551 766 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785992227050689-9601-0/minicluster-data/ts-0/data/info.pb
I20260806 05:01:09.685354 9601 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu as pid 766
I20260806 05:01:09.685618 9601 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785992227050689-9601-0/minicluster-data/ts-0/wal/instance
I20260806 05:01:09.686276 9601 external_mini_cluster.cc:1428] Setting key 1b0259a43454df08a037801cdab2f8de
Aug 06 05:01:09 dist-test-slave-x4qh krb5kdc[593](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992469, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.96.65@KRBTEST.COM for kudu/127.9.96.126@KRBTEST.COM
I20260806 05:01:09.700114 951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:01:09.681967 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.96.65:53153 (local address 127.9.96.126:34193)
0806 05:01:09.682467 (+ 500us) server_negotiation.cc:206] Beginning negotiation
0806 05:01:09.682472 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 05:01:09.683232 (+ 760us) server_negotiation.cc:407] Connection header received
0806 05:01:09.685106 (+ 1874us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 05:01:09.685112 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 05:01:09.685199 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 05:01:09.685339 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0806 05:01:09.686892 (+ 1553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:01:09.687881 (+ 989us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:01:09.689434 (+ 1553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:01:09.689845 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:01:09.689982 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:01:09.693926 (+ 3944us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 05:01:09.693930 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 05:01:09.693934 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 05:01:09.693965 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0806 05:01:09.696540 (+ 2575us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 05:01:09.697487 (+ 947us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 05:01:09.697491 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 05:01:09.697494 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 05:01:09.697611 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 05:01:09.698091 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 05:01:09.698094 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 05:01:09.698096 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 05:01:09.698338 (+ 242us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 05:01:09.698447 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0806 05:01:09.699390 (+ 943us) server_negotiation.cc:299] Negotiation successful
0806 05:01:09.699645 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":389,"thread_start_us":267,"threads_started":1}
I20260806 05:01:09.700707 955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:01:09.682372 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:34193 (local address 127.9.96.65:53153)
0806 05:01:09.683079 (+ 707us) negotiation.cc:107] Waiting for socket to connect
0806 05:01:09.683115 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0806 05:01:09.684846 (+ 1731us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 05:01:09.685576 (+ 730us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 05:01:09.685584 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 05:01:09.686121 (+ 537us) client_negotiation.cc:189] Negotiated authn=SASL
0806 05:01:09.686718 (+ 597us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 05:01:09.686734 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 05:01:09.688084 (+ 1350us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 05:01:09.688088 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 05:01:09.689283 (+ 1195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 05:01:09.689295 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 05:01:09.690090 (+ 795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 05:01:09.690098 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 05:01:09.690331 (+ 233us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:01:09.690992 (+ 661us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 05:01:09.691018 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0806 05:01:09.693708 (+ 2690us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 05:01:09.696677 (+ 2969us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 05:01:09.696688 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 05:01:09.696705 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0806 05:01:09.697365 (+ 660us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 05:01:09.697720 (+ 355us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 05:01:09.697727 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 05:01:09.697733 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0806 05:01:09.697982 (+ 249us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 05:01:09.698465 (+ 483us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 05:01:09.698475 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 05:01:09.698919 (+ 444us) client_negotiation.cc:769] Sending connection context
0806 05:01:09.699232 (+ 313us) client_negotiation.cc:240] Negotiation successful
0806 05:01:09.699666 (+ 434us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":491,"thread_start_us":295,"threads_started":1}
I20260806 05:01:09.702201 949 heartbeater.cc:347] Connected to a master server at 127.9.96.126:34193
I20260806 05:01:09.702491 949 heartbeater.cc:464] Registering TS with master...
I20260806 05:01:09.703190 949 heartbeater.cc:511] Master 127.9.96.126:34193 requested a full tablet report, sending...
I20260806 05:01:09.705295 668 ts_manager.cc:194] Registered new tserver with Master: 14be68f793794acaba3556a8e1ef3c13 (127.9.96.65:45547)
I20260806 05:01:09.706713 668 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.96.65@KRBTEST.COM'} at 127.9.96.65:53153
WARNING: no policy specified for kudu/127.9.96.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.96.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.96.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.66.keytab.
Entry for principal kudu/127.9.96.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.66.keytab.
I20260806 05:01:09.748669 9601 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
/tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785992227050689-9601-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785992227050689-9601-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785992227050689-9601-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785992227050689-9601-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.96.66:0
--local_ip_for_outbound_sockets=127.9.96.66
--webserver_interface=127.9.96.66
--webserver_port=0
--tserver_master_addrs=127.9.96.126:34193
--keytab_file=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.66.keytab
--principal=kudu/127.9.96.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.96.84:41431
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785992227050689-9601-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785992227050689-9601-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785992227050689-9601-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 05:01:10.078022 960 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 05:01:10.078277 960 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 05:01:10.078437 960 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 05:01:10.085908 960 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 05:01:10.086061 960 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 05:01:10.086246 960 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.96.66
I20260806 05:01:10.095583 960 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.96.84:41431
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785992227050689-9601-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785992227050689-9601-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.66.keytab
--principal=kudu/127.9.96.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.96.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785992227050689-9601-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.96.66
--webserver_port=0
--tserver_master_addrs=127.9.96.126:34193
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.96.66
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785992227050689-9601-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 03:56:12 UTC on ccycloud.quasar-guijbm.root.comops.site
build id 4569618
ASAN enabled
I20260806 05:01:10.096773 960 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 05:01:10.098105 960 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 05:01:10.111186 969 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 05:01:10.111236 972 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 05:01:10.111271 974 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 05:01:10.112089 960 server_base.cc:1034] running on GCE node
I20260806 05:01:10.112635 960 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 05:01:10.113471 960 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 05:01:10.114665 960 hybrid_clock.cc:648] HybridClock initialized: now 1785992470114628 us; error 33 us; skew 500 ppm
I20260806 05:01:10.115032 960 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 05:01:10 dist-test-slave-x4qh krb5kdc[593](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992470, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.96.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 05:01:10.118314 960 init.cc:377] Logged in from keytab as kudu/127.9.96.66@KRBTEST.COM (short username kudu)
I20260806 05:01:10.119902 960 webserver.cc:460] Webserver started at http://127.9.96.66:37521/ using document root <none> and password file <none>
I20260806 05:01:10.120656 960 fs_manager.cc:359] Metadata directory not provided
I20260806 05:01:10.120805 960 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 05:01:10.121124 960 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 05:01:10.124408 960 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785992227050689-9601-0/minicluster-data/ts-1/data/instance:
uuid: "2c79466ec5a14051bbc556f5a3852645"
format_stamp: "Formatted at 2026-08-06 05:01:10 on dist-test-slave-x4qh"
server_key: "9e64761b3a121e6ad69b6aa80e61efae"
server_key_iv: "7a1c3766778b4209965db7e3fa55f2ef"
server_key_version: "encryptionkey@0"
I20260806 05:01:10.125171 960 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785992227050689-9601-0/minicluster-data/ts-1/wal/instance:
uuid: "2c79466ec5a14051bbc556f5a3852645"
format_stamp: "Formatted at 2026-08-06 05:01:10 on dist-test-slave-x4qh"
server_key: "9e64761b3a121e6ad69b6aa80e61efae"
server_key_iv: "7a1c3766778b4209965db7e3fa55f2ef"
server_key_version: "encryptionkey@0"
I20260806 05:01:10.130326 960 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.005s
I20260806 05:01:10.133625 987 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 05:01:10.134914 960 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260806 05:01:10.135084 960 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785992227050689-9601-0/minicluster-data/ts-1/data,/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785992227050689-9601-0/minicluster-data/ts-1/wal
uuid: "2c79466ec5a14051bbc556f5a3852645"
format_stamp: "Formatted at 2026-08-06 05:01:10 on dist-test-slave-x4qh"
server_key: "9e64761b3a121e6ad69b6aa80e61efae"
server_key_iv: "7a1c3766778b4209965db7e3fa55f2ef"
server_key_version: "encryptionkey@0"
I20260806 05:01:10.135280 960 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785992227050689-9601-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785992227050689-9601-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785992227050689-9601-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 05:01:10.150015 960 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 05:01:10.154280 960 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 05:01:10.154587 960 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 05:01:10.155611 960 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 05:01:10.156878 960 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 05:01:10.156983 960 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 05:01:10.157128 960 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 05:01:10.157209 960 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 05:01:10.191923 960 rpc_server.cc:307] RPC server started. Bound to: 127.9.96.66:36483
I20260806 05:01:10.191982 1165 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.96.66:36483 every 8 connection(s)
I20260806 05:01:10.193224 960 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785992227050689-9601-0/minicluster-data/ts-1/data/info.pb
I20260806 05:01:10.203469 9601 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu as pid 960
I20260806 05:01:10.203671 9601 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785992227050689-9601-0/minicluster-data/ts-1/wal/instance
I20260806 05:01:10.204274 9601 external_mini_cluster.cc:1428] Setting key b44e5c3110383440fcb14082244bc584
Aug 06 05:01:10 dist-test-slave-x4qh krb5kdc[593](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992470, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.96.66@KRBTEST.COM for kudu/127.9.96.126@KRBTEST.COM
I20260806 05:01:10.213331 951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:01:10.195738 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.96.66:54255 (local address 127.9.96.126:34193)
0806 05:01:10.195917 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0806 05:01:10.195922 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 05:01:10.197088 (+ 1166us) server_negotiation.cc:407] Connection header received
0806 05:01:10.198828 (+ 1740us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 05:01:10.198833 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 05:01:10.198905 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 05:01:10.198991 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0806 05:01:10.200459 (+ 1468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:01:10.201167 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:01:10.202733 (+ 1566us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:01:10.203144 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:01:10.203234 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:01:10.207465 (+ 4231us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 05:01:10.207470 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 05:01:10.207473 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 05:01:10.207505 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0806 05:01:10.210147 (+ 2642us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 05:01:10.211055 (+ 908us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 05:01:10.211059 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 05:01:10.211061 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 05:01:10.211184 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 05:01:10.211694 (+ 510us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 05:01:10.211697 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 05:01:10.211699 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 05:01:10.211961 (+ 262us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 05:01:10.212071 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0806 05:01:10.212938 (+ 867us) server_negotiation.cc:299] Negotiation successful
0806 05:01:10.213101 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260806 05:01:10.214787 1169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:01:10.196229 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:34193 (local address 127.9.96.66:54255)
0806 05:01:10.196928 (+ 699us) negotiation.cc:107] Waiting for socket to connect
0806 05:01:10.196968 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0806 05:01:10.198592 (+ 1624us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 05:01:10.199184 (+ 592us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 05:01:10.199191 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 05:01:10.199733 (+ 542us) client_negotiation.cc:189] Negotiated authn=SASL
0806 05:01:10.200298 (+ 565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 05:01:10.200325 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 05:01:10.201332 (+ 1007us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 05:01:10.201340 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 05:01:10.202610 (+ 1270us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 05:01:10.202622 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 05:01:10.203402 (+ 780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 05:01:10.203411 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 05:01:10.203735 (+ 324us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:01:10.204451 (+ 716us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 05:01:10.204481 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0806 05:01:10.207198 (+ 2717us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 05:01:10.210286 (+ 3088us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 05:01:10.210297 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 05:01:10.210314 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0806 05:01:10.210932 (+ 618us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 05:01:10.211292 (+ 360us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 05:01:10.211298 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 05:01:10.211303 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 05:01:10.211573 (+ 270us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 05:01:10.212090 (+ 517us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 05:01:10.212100 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 05:01:10.212535 (+ 435us) client_negotiation.cc:769] Sending connection context
0806 05:01:10.212850 (+ 315us) client_negotiation.cc:240] Negotiation successful
0806 05:01:10.213386 (+ 536us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":485,"thread_start_us":281,"threads_started":1}
I20260806 05:01:10.216248 1166 heartbeater.cc:347] Connected to a master server at 127.9.96.126:34193
I20260806 05:01:10.216572 1166 heartbeater.cc:464] Registering TS with master...
I20260806 05:01:10.217267 1166 heartbeater.cc:511] Master 127.9.96.126:34193 requested a full tablet report, sending...
I20260806 05:01:10.218621 668 ts_manager.cc:194] Registered new tserver with Master: 2c79466ec5a14051bbc556f5a3852645 (127.9.96.66:36483)
I20260806 05:01:10.219473 668 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.96.66@KRBTEST.COM'} at 127.9.96.66:54255
WARNING: no policy specified for kudu/127.9.96.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.96.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.96.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.67.keytab.
Entry for principal kudu/127.9.96.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.67.keytab.
I20260806 05:01:10.264882 9601 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
/tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785992227050689-9601-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785992227050689-9601-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785992227050689-9601-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785992227050689-9601-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.96.67:0
--local_ip_for_outbound_sockets=127.9.96.67
--webserver_interface=127.9.96.67
--webserver_port=0
--tserver_master_addrs=127.9.96.126:34193
--keytab_file=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.67.keytab
--principal=kudu/127.9.96.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.96.84:41431
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785992227050689-9601-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785992227050689-9601-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785992227050689-9601-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 05:01:10.590822 1174 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 05:01:10.591121 1174 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 05:01:10.591296 1174 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 05:01:10.598395 1174 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 05:01:10.598526 1174 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 05:01:10.598704 1174 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.96.67
I20260806 05:01:10.608111 1174 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.96.84:41431
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785992227050689-9601-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785992227050689-9601-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.67.keytab
--principal=kudu/127.9.96.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.96.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785992227050689-9601-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.96.67
--webserver_port=0
--tserver_master_addrs=127.9.96.126:34193
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.96.67
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785992227050689-9601-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 03:56:12 UTC on ccycloud.quasar-guijbm.root.comops.site
build id 4569618
ASAN enabled
I20260806 05:01:10.609372 1174 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 05:01:10.610690 1174 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 05:01:10.623931 1181 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 05:01:10.623890 1186 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 05:01:10.623973 1180 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 05:01:10.625350 1174 server_base.cc:1034] running on GCE node
I20260806 05:01:10.626031 1174 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 05:01:10.626940 1174 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 05:01:10.628183 1174 hybrid_clock.cc:648] HybridClock initialized: now 1785992470628122 us; error 49 us; skew 500 ppm
I20260806 05:01:10.628496 1174 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 05:01:10 dist-test-slave-x4qh krb5kdc[593](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992470, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.96.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 05:01:10.631584 1174 init.cc:377] Logged in from keytab as kudu/127.9.96.67@KRBTEST.COM (short username kudu)
I20260806 05:01:10.633073 1174 webserver.cc:460] Webserver started at http://127.9.96.67:33175/ using document root <none> and password file <none>
I20260806 05:01:10.633730 1174 fs_manager.cc:359] Metadata directory not provided
I20260806 05:01:10.633831 1174 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 05:01:10.634099 1174 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 05:01:10.636953 1174 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785992227050689-9601-0/minicluster-data/ts-2/data/instance:
uuid: "6ebbb98128aa4bd69728676853db94a2"
format_stamp: "Formatted at 2026-08-06 05:01:10 on dist-test-slave-x4qh"
server_key: "696c825852d11aca183a8bfa31ff49da"
server_key_iv: "008ddaa8c2b7b319919f9c88a012add6"
server_key_version: "encryptionkey@0"
I20260806 05:01:10.637624 1174 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785992227050689-9601-0/minicluster-data/ts-2/wal/instance:
uuid: "6ebbb98128aa4bd69728676853db94a2"
format_stamp: "Formatted at 2026-08-06 05:01:10 on dist-test-slave-x4qh"
server_key: "696c825852d11aca183a8bfa31ff49da"
server_key_iv: "008ddaa8c2b7b319919f9c88a012add6"
server_key_version: "encryptionkey@0"
I20260806 05:01:10.643036 1174 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260806 05:01:10.646417 1196 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 05:01:10.647786 1174 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 05:01:10.647975 1174 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785992227050689-9601-0/minicluster-data/ts-2/data,/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785992227050689-9601-0/minicluster-data/ts-2/wal
uuid: "6ebbb98128aa4bd69728676853db94a2"
format_stamp: "Formatted at 2026-08-06 05:01:10 on dist-test-slave-x4qh"
server_key: "696c825852d11aca183a8bfa31ff49da"
server_key_iv: "008ddaa8c2b7b319919f9c88a012add6"
server_key_version: "encryptionkey@0"
I20260806 05:01:10.648149 1174 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785992227050689-9601-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785992227050689-9601-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785992227050689-9601-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 05:01:10.660923 1174 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 05:01:10.664878 1174 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 05:01:10.665151 1174 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 05:01:10.666086 1174 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 05:01:10.667332 1174 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 05:01:10.667434 1174 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 05:01:10.667519 1174 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 05:01:10.667567 1174 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 05:01:10.701908 1174 rpc_server.cc:307] RPC server started. Bound to: 127.9.96.67:41539
I20260806 05:01:10.701951 1365 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.96.67:41539 every 8 connection(s)
I20260806 05:01:10.703109 1174 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785992227050689-9601-0/minicluster-data/ts-2/data/info.pb
I20260806 05:01:10.709223 9601 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu as pid 1174
I20260806 05:01:10.709427 9601 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785992227050689-9601-0/minicluster-data/ts-2/wal/instance
I20260806 05:01:10.710042 9601 external_mini_cluster.cc:1428] Setting key 4346a87278fb30e03210a1d01bd563f0
I20260806 05:01:10.710237 949 heartbeater.cc:503] Master 127.9.96.126:34193 was elected leader, sending a full tablet report...
Aug 06 05:01:10 dist-test-slave-x4qh krb5kdc[593](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992470, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.96.67@KRBTEST.COM for kudu/127.9.96.126@KRBTEST.COM
I20260806 05:01:10.724166 951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:01:10.705705 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.96.67:56649 (local address 127.9.96.126:34193)
0806 05:01:10.705921 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0806 05:01:10.705930 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 05:01:10.707008 (+ 1078us) server_negotiation.cc:407] Connection header received
0806 05:01:10.708966 (+ 1958us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 05:01:10.708971 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 05:01:10.709040 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 05:01:10.709128 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0806 05:01:10.711147 (+ 2019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:01:10.712501 (+ 1354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:01:10.714404 (+ 1903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:01:10.714817 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:01:10.714977 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:01:10.718580 (+ 3603us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 05:01:10.718584 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 05:01:10.718587 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 05:01:10.718617 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0806 05:01:10.721174 (+ 2557us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 05:01:10.721888 (+ 714us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 05:01:10.721899 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 05:01:10.721903 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 05:01:10.722026 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 05:01:10.722445 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 05:01:10.722451 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 05:01:10.722454 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 05:01:10.722715 (+ 261us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 05:01:10.722878 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0806 05:01:10.723590 (+ 712us) server_negotiation.cc:299] Negotiation successful
0806 05:01:10.723825 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260806 05:01:10.725246 1369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:01:10.706134 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:34193 (local address 127.9.96.67:56649)
0806 05:01:10.706858 (+ 724us) negotiation.cc:107] Waiting for socket to connect
0806 05:01:10.706893 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0806 05:01:10.708668 (+ 1775us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 05:01:10.709408 (+ 740us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 05:01:10.709423 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 05:01:10.710176 (+ 753us) client_negotiation.cc:189] Negotiated authn=SASL
0806 05:01:10.710935 (+ 759us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 05:01:10.710983 (+ 48us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 05:01:10.712663 (+ 1680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 05:01:10.712669 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 05:01:10.714253 (+ 1584us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 05:01:10.714272 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 05:01:10.715062 (+ 790us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 05:01:10.715072 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 05:01:10.715461 (+ 389us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:01:10.716251 (+ 790us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 05:01:10.716273 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0806 05:01:10.718360 (+ 2087us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 05:01:10.721328 (+ 2968us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 05:01:10.721337 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 05:01:10.721351 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0806 05:01:10.721759 (+ 408us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 05:01:10.722138 (+ 379us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 05:01:10.722144 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 05:01:10.722147 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 05:01:10.722325 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 05:01:10.722843 (+ 518us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 05:01:10.722850 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 05:01:10.723173 (+ 323us) client_negotiation.cc:769] Sending connection context
0806 05:01:10.723470 (+ 297us) client_negotiation.cc:240] Negotiation successful
0806 05:01:10.723950 (+ 480us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":509,"thread_start_us":282,"threads_started":1}
I20260806 05:01:10.726649 1367 heartbeater.cc:347] Connected to a master server at 127.9.96.126:34193
I20260806 05:01:10.726936 1367 heartbeater.cc:464] Registering TS with master...
I20260806 05:01:10.727644 1367 heartbeater.cc:511] Master 127.9.96.126:34193 requested a full tablet report, sending...
I20260806 05:01:10.728935 667 ts_manager.cc:194] Registered new tserver with Master: 6ebbb98128aa4bd69728676853db94a2 (127.9.96.67:41539)
I20260806 05:01:10.729795 667 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.96.67@KRBTEST.COM'} at 127.9.96.67:56649
I20260806 05:01:10.738036 9601 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 05:01:10.754572 951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:01:10.742723 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43684 (local address 127.9.96.126:34193)
0806 05:01:10.742890 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0806 05:01:10.742894 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 05:01:10.743312 (+ 418us) server_negotiation.cc:407] Connection header received
0806 05:01:10.743503 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 05:01:10.743506 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 05:01:10.743575 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 05:01:10.743652 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0806 05:01:10.745047 (+ 1395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:01:10.745754 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:01:10.747069 (+ 1315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:01:10.747516 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:01:10.747605 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:01:10.749958 (+ 2353us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 05:01:10.749963 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 05:01:10.749966 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 05:01:10.749995 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0806 05:01:10.752378 (+ 2383us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 05:01:10.753023 (+ 645us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 05:01:10.753027 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 05:01:10.753029 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 05:01:10.753152 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 05:01:10.753471 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 05:01:10.753475 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 05:01:10.753476 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 05:01:10.753760 (+ 284us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 05:01:10.753943 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0806 05:01:10.754202 (+ 259us) server_negotiation.cc:299] Negotiation successful
0806 05:01:10.754354 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260806 05:01:10.758356 667 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43684:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 05:01:10.765021 667 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 05:01:10.789299 1386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:01:10.776482 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.66:36483 (local address 127.0.0.1:43694)
0806 05:01:10.778184 (+ 1702us) negotiation.cc:107] Waiting for socket to connect
0806 05:01:10.778245 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0806 05:01:10.778469 (+ 224us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 05:01:10.780680 (+ 2211us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 05:01:10.780694 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 05:01:10.780741 (+ 47us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 05:01:10.781271 (+ 530us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 05:01:10.781288 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 05:01:10.783665 (+ 2377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 05:01:10.783670 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 05:01:10.786231 (+ 2561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 05:01:10.786247 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 05:01:10.788204 (+ 1957us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 05:01:10.788211 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 05:01:10.788441 (+ 230us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:01:10.788498 (+ 57us) client_negotiation.cc:769] Sending connection context
0806 05:01:10.788603 (+ 105us) client_negotiation.cc:240] Negotiation successful
0806 05:01:10.788784 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1382,"mutex_wait_us":110,"thread_start_us":4310,"threads_started":1}
I20260806 05:01:10.789898 1387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:01:10.777709 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56390 (local address 127.9.96.65:45547)
0806 05:01:10.779286 (+ 1577us) server_negotiation.cc:206] Beginning negotiation
0806 05:01:10.779298 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 05:01:10.781306 (+ 2008us) server_negotiation.cc:407] Connection header received
0806 05:01:10.781637 (+ 331us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 05:01:10.781648 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 05:01:10.781952 (+ 304us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 05:01:10.782175 (+ 223us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 05:01:10.782965 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:01:10.784629 (+ 1664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:01:10.786998 (+ 2369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:01:10.788497 (+ 1499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:01:10.788646 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:01:10.788888 (+ 242us) server_negotiation.cc:1035] Waiting for connection context
0806 05:01:10.789383 (+ 495us) server_negotiation.cc:299] Negotiation successful
0806 05:01:10.789551 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1352,"thread_start_us":1135,"threads_started":1}
I20260806 05:01:10.789950 1389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:01:10.779952 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.67:41539 (local address 127.0.0.1:57950)
0806 05:01:10.781564 (+ 1612us) negotiation.cc:107] Waiting for socket to connect
0806 05:01:10.781581 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 05:01:10.781680 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 05:01:10.782881 (+ 1201us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 05:01:10.782889 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 05:01:10.782915 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 05:01:10.783294 (+ 379us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 05:01:10.783308 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 05:01:10.785199 (+ 1891us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 05:01:10.785206 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 05:01:10.787275 (+ 2069us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 05:01:10.787291 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 05:01:10.789160 (+ 1869us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 05:01:10.789166 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 05:01:10.789333 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:01:10.789362 (+ 29us) client_negotiation.cc:769] Sending connection context
0806 05:01:10.789447 (+ 85us) client_negotiation.cc:240] Negotiation successful
0806 05:01:10.789560 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1501,"mutex_wait_us":20,"thread_start_us":1230,"threads_started":1}
I20260806 05:01:10.790143 1391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:01:10.780667 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57950 (local address 127.9.96.67:41539)
0806 05:01:10.782302 (+ 1635us) server_negotiation.cc:206] Beginning negotiation
0806 05:01:10.782310 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 05:01:10.782341 (+ 31us) server_negotiation.cc:407] Connection header received
0806 05:01:10.782430 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 05:01:10.782437 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 05:01:10.782634 (+ 197us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 05:01:10.782854 (+ 220us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 05:01:10.783547 (+ 693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:01:10.785059 (+ 1512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:01:10.787489 (+ 2430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:01:10.788933 (+ 1444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:01:10.789082 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:01:10.789528 (+ 446us) server_negotiation.cc:1035] Waiting for connection context
0806 05:01:10.789605 (+ 77us) server_negotiation.cc:299] Negotiation successful
0806 05:01:10.789763 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1443,"thread_start_us":1248,"threads_started":1}
I20260806 05:01:10.790428 1384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:01:10.776945 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.65:45547 (local address 127.0.0.1:56390)
0806 05:01:10.781123 (+ 4178us) negotiation.cc:107] Waiting for socket to connect
0806 05:01:10.781148 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0806 05:01:10.781369 (+ 221us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 05:01:10.782375 (+ 1006us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 05:01:10.782378 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 05:01:10.782398 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 05:01:10.782689 (+ 291us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 05:01:10.782696 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 05:01:10.784781 (+ 2085us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 05:01:10.784787 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 05:01:10.786846 (+ 2059us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 05:01:10.786861 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 05:01:10.788720 (+ 1859us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 05:01:10.788727 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 05:01:10.788919 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:01:10.788955 (+ 36us) client_negotiation.cc:769] Sending connection context
0806 05:01:10.789061 (+ 106us) client_negotiation.cc:240] Negotiation successful
0806 05:01:10.790107 (+ 1046us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4012,"mutex_wait_us":24,"thread_start_us":923,"threads_started":1}
I20260806 05:01:10.791345 1385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:01:10.776690 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43694 (local address 127.9.96.66:36483)
0806 05:01:10.779906 (+ 3216us) server_negotiation.cc:206] Beginning negotiation
0806 05:01:10.779915 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 05:01:10.779944 (+ 29us) server_negotiation.cc:407] Connection header received
0806 05:01:10.780035 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 05:01:10.780043 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 05:01:10.780316 (+ 273us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 05:01:10.781532 (+ 1216us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 05:01:10.781768 (+ 236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:01:10.783515 (+ 1747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:01:10.786402 (+ 2887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:01:10.787966 (+ 1564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:01:10.790676 (+ 2710us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:01:10.790841 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0806 05:01:10.790902 (+ 61us) server_negotiation.cc:299] Negotiation successful
0806 05:01:10.791056 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3063,"thread_start_us":2795,"threads_started":1}
I20260806 05:01:10.793502 854 tablet_service.cc:1467] Processing CreateTablet for tablet c2e4ae9e26b142abb7c57c69dd511f51 (DEFAULT_TABLE table=test-table [id=53f5e000f00c48d5af881d35d00ad61e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 05:01:10.793632 1266 tablet_service.cc:1467] Processing CreateTablet for tablet c2e4ae9e26b142abb7c57c69dd511f51 (DEFAULT_TABLE table=test-table [id=53f5e000f00c48d5af881d35d00ad61e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 05:01:10.794849 1266 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c2e4ae9e26b142abb7c57c69dd511f51. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 05:01:10.794914 854 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c2e4ae9e26b142abb7c57c69dd511f51. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 05:01:10.796576 1057 tablet_service.cc:1467] Processing CreateTablet for tablet c2e4ae9e26b142abb7c57c69dd511f51 (DEFAULT_TABLE table=test-table [id=53f5e000f00c48d5af881d35d00ad61e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 05:01:10.797828 1057 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c2e4ae9e26b142abb7c57c69dd511f51. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 05:01:10.806881 1392 tablet_bootstrap.cc:492] T c2e4ae9e26b142abb7c57c69dd511f51 P 6ebbb98128aa4bd69728676853db94a2: Bootstrap starting.
I20260806 05:01:10.808826 1393 tablet_bootstrap.cc:492] T c2e4ae9e26b142abb7c57c69dd511f51 P 14be68f793794acaba3556a8e1ef3c13: Bootstrap starting.
I20260806 05:01:10.810537 1392 tablet_bootstrap.cc:654] T c2e4ae9e26b142abb7c57c69dd511f51 P 6ebbb98128aa4bd69728676853db94a2: Neither blocks nor log segments found. Creating new log.
I20260806 05:01:10.811164 1394 tablet_bootstrap.cc:492] T c2e4ae9e26b142abb7c57c69dd511f51 P 2c79466ec5a14051bbc556f5a3852645: Bootstrap starting.
I20260806 05:01:10.811947 1392 log.cc:824] T c2e4ae9e26b142abb7c57c69dd511f51 P 6ebbb98128aa4bd69728676853db94a2: Log is configured to *not* fsync() on all Append() calls
I20260806 05:01:10.812549 1393 tablet_bootstrap.cc:654] T c2e4ae9e26b142abb7c57c69dd511f51 P 14be68f793794acaba3556a8e1ef3c13: Neither blocks nor log segments found. Creating new log.
I20260806 05:01:10.813675 1393 log.cc:824] T c2e4ae9e26b142abb7c57c69dd511f51 P 14be68f793794acaba3556a8e1ef3c13: Log is configured to *not* fsync() on all Append() calls
I20260806 05:01:10.814291 1394 tablet_bootstrap.cc:654] T c2e4ae9e26b142abb7c57c69dd511f51 P 2c79466ec5a14051bbc556f5a3852645: Neither blocks nor log segments found. Creating new log.
I20260806 05:01:10.815701 1392 tablet_bootstrap.cc:492] T c2e4ae9e26b142abb7c57c69dd511f51 P 6ebbb98128aa4bd69728676853db94a2: No bootstrap required, opened a new log
I20260806 05:01:10.815743 1394 log.cc:824] T c2e4ae9e26b142abb7c57c69dd511f51 P 2c79466ec5a14051bbc556f5a3852645: Log is configured to *not* fsync() on all Append() calls
I20260806 05:01:10.815992 1392 ts_tablet_manager.cc:1397] T c2e4ae9e26b142abb7c57c69dd511f51 P 6ebbb98128aa4bd69728676853db94a2: Time spent bootstrapping tablet: real 0.009s user 0.008s sys 0.000s
I20260806 05:01:10.817643 1393 tablet_bootstrap.cc:492] T c2e4ae9e26b142abb7c57c69dd511f51 P 14be68f793794acaba3556a8e1ef3c13: No bootstrap required, opened a new log
I20260806 05:01:10.817962 1393 ts_tablet_manager.cc:1397] T c2e4ae9e26b142abb7c57c69dd511f51 P 14be68f793794acaba3556a8e1ef3c13: Time spent bootstrapping tablet: real 0.009s user 0.008s sys 0.000s
I20260806 05:01:10.818451 1394 tablet_bootstrap.cc:492] T c2e4ae9e26b142abb7c57c69dd511f51 P 2c79466ec5a14051bbc556f5a3852645: No bootstrap required, opened a new log
I20260806 05:01:10.818711 1394 ts_tablet_manager.cc:1397] T c2e4ae9e26b142abb7c57c69dd511f51 P 2c79466ec5a14051bbc556f5a3852645: Time spent bootstrapping tablet: real 0.008s user 0.003s sys 0.003s
I20260806 05:01:10.826553 1394 raft_consensus.cc:348] T c2e4ae9e26b142abb7c57c69dd511f51 P 2c79466ec5a14051bbc556f5a3852645 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c79466ec5a14051bbc556f5a3852645" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 36483 } } peers { permanent_uuid: "14be68f793794acaba3556a8e1ef3c13" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 45547 } } peers { permanent_uuid: "6ebbb98128aa4bd69728676853db94a2" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 41539 } }
I20260806 05:01:10.827021 1394 raft_consensus.cc:374] T c2e4ae9e26b142abb7c57c69dd511f51 P 2c79466ec5a14051bbc556f5a3852645 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 05:01:10.827153 1394 raft_consensus.cc:729] T c2e4ae9e26b142abb7c57c69dd511f51 P 2c79466ec5a14051bbc556f5a3852645 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c79466ec5a14051bbc556f5a3852645, State: Initialized, Role: FOLLOWER
I20260806 05:01:10.826893 1392 raft_consensus.cc:348] T c2e4ae9e26b142abb7c57c69dd511f51 P 6ebbb98128aa4bd69728676853db94a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c79466ec5a14051bbc556f5a3852645" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 36483 } } peers { permanent_uuid: "14be68f793794acaba3556a8e1ef3c13" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 45547 } } peers { permanent_uuid: "6ebbb98128aa4bd69728676853db94a2" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 41539 } }
I20260806 05:01:10.827301 1392 raft_consensus.cc:374] T c2e4ae9e26b142abb7c57c69dd511f51 P 6ebbb98128aa4bd69728676853db94a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 05:01:10.827414 1392 raft_consensus.cc:729] T c2e4ae9e26b142abb7c57c69dd511f51 P 6ebbb98128aa4bd69728676853db94a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6ebbb98128aa4bd69728676853db94a2, State: Initialized, Role: FOLLOWER
I20260806 05:01:10.827808 1394 consensus_queue.cc:260] T c2e4ae9e26b142abb7c57c69dd511f51 P 2c79466ec5a14051bbc556f5a3852645 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c79466ec5a14051bbc556f5a3852645" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 36483 } } peers { permanent_uuid: "14be68f793794acaba3556a8e1ef3c13" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 45547 } } peers { permanent_uuid: "6ebbb98128aa4bd69728676853db94a2" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 41539 } }
I20260806 05:01:10.828060 1392 consensus_queue.cc:260] T c2e4ae9e26b142abb7c57c69dd511f51 P 6ebbb98128aa4bd69728676853db94a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c79466ec5a14051bbc556f5a3852645" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 36483 } } peers { permanent_uuid: "14be68f793794acaba3556a8e1ef3c13" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 45547 } } peers { permanent_uuid: "6ebbb98128aa4bd69728676853db94a2" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 41539 } }
I20260806 05:01:10.828521 1393 raft_consensus.cc:348] T c2e4ae9e26b142abb7c57c69dd511f51 P 14be68f793794acaba3556a8e1ef3c13 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c79466ec5a14051bbc556f5a3852645" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 36483 } } peers { permanent_uuid: "14be68f793794acaba3556a8e1ef3c13" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 45547 } } peers { permanent_uuid: "6ebbb98128aa4bd69728676853db94a2" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 41539 } }
I20260806 05:01:10.828938 1166 heartbeater.cc:503] Master 127.9.96.126:34193 was elected leader, sending a full tablet report...
I20260806 05:01:10.829088 1393 raft_consensus.cc:374] T c2e4ae9e26b142abb7c57c69dd511f51 P 14be68f793794acaba3556a8e1ef3c13 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 05:01:10.829198 1393 raft_consensus.cc:729] T c2e4ae9e26b142abb7c57c69dd511f51 P 14be68f793794acaba3556a8e1ef3c13 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 14be68f793794acaba3556a8e1ef3c13, State: Initialized, Role: FOLLOWER
I20260806 05:01:10.829684 1394 ts_tablet_manager.cc:1428] T c2e4ae9e26b142abb7c57c69dd511f51 P 2c79466ec5a14051bbc556f5a3852645: Time spent starting tablet: real 0.011s user 0.011s sys 0.000s
I20260806 05:01:10.830009 1393 consensus_queue.cc:260] T c2e4ae9e26b142abb7c57c69dd511f51 P 14be68f793794acaba3556a8e1ef3c13 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c79466ec5a14051bbc556f5a3852645" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 36483 } } peers { permanent_uuid: "14be68f793794acaba3556a8e1ef3c13" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 45547 } } peers { permanent_uuid: "6ebbb98128aa4bd69728676853db94a2" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 41539 } }
I20260806 05:01:10.830681 1367 heartbeater.cc:503] Master 127.9.96.126:34193 was elected leader, sending a full tablet report...
I20260806 05:01:10.831528 1392 ts_tablet_manager.cc:1428] T c2e4ae9e26b142abb7c57c69dd511f51 P 6ebbb98128aa4bd69728676853db94a2: Time spent starting tablet: real 0.015s user 0.016s sys 0.000s
I20260806 05:01:10.833217 1393 ts_tablet_manager.cc:1428] T c2e4ae9e26b142abb7c57c69dd511f51 P 14be68f793794acaba3556a8e1ef3c13: Time spent starting tablet: real 0.015s user 0.007s sys 0.007s
W20260806 05:01:10.932219 950 tablet.cc:2369] T c2e4ae9e26b142abb7c57c69dd511f51 P 14be68f793794acaba3556a8e1ef3c13: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 05:01:10.945300 1167 tablet.cc:2369] T c2e4ae9e26b142abb7c57c69dd511f51 P 2c79466ec5a14051bbc556f5a3852645: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 05:01:10.954881 1368 tablet.cc:2369] T c2e4ae9e26b142abb7c57c69dd511f51 P 6ebbb98128aa4bd69728676853db94a2: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 05:01:11.031720 1399 raft_consensus.cc:482] T c2e4ae9e26b142abb7c57c69dd511f51 P 2c79466ec5a14051bbc556f5a3852645 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 05:01:11.032063 1399 raft_consensus.cc:504] T c2e4ae9e26b142abb7c57c69dd511f51 P 2c79466ec5a14051bbc556f5a3852645 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c79466ec5a14051bbc556f5a3852645" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 36483 } } peers { permanent_uuid: "14be68f793794acaba3556a8e1ef3c13" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 45547 } } peers { permanent_uuid: "6ebbb98128aa4bd69728676853db94a2" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 41539 } }
I20260806 05:01:11.033701 1399 leader_election.cc:288] T c2e4ae9e26b142abb7c57c69dd511f51 P 2c79466ec5a14051bbc556f5a3852645 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 14be68f793794acaba3556a8e1ef3c13 (127.9.96.65:45547), 6ebbb98128aa4bd69728676853db94a2 (127.9.96.67:41539)
I20260806 05:01:11.042330 1387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:01:11.034649 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.96.66:57849 (local address 127.9.96.65:45547)
0806 05:01:11.034861 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0806 05:01:11.034867 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 05:01:11.035122 (+ 255us) server_negotiation.cc:407] Connection header received
0806 05:01:11.035516 (+ 394us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 05:01:11.035522 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 05:01:11.035605 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 05:01:11.035675 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 05:01:11.036542 (+ 867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:01:11.037904 (+ 1362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:01:11.040407 (+ 2503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:01:11.041377 (+ 970us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:01:11.041526 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:01:11.041640 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0806 05:01:11.041937 (+ 297us) server_negotiation.cc:299] Negotiation successful
0806 05:01:11.042067 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260806 05:01:11.042398 1404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:01:11.033997 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.65:45547 (local address 127.9.96.66:57849)
0806 05:01:11.035009 (+ 1012us) negotiation.cc:107] Waiting for socket to connect
0806 05:01:11.035029 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0806 05:01:11.035273 (+ 244us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 05:01:11.035894 (+ 621us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 05:01:11.035897 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 05:01:11.035917 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 05:01:11.036355 (+ 438us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 05:01:11.036369 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 05:01:11.038065 (+ 1696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 05:01:11.038071 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 05:01:11.040198 (+ 2127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 05:01:11.040209 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 05:01:11.041599 (+ 1390us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 05:01:11.041605 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 05:01:11.041770 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:01:11.041794 (+ 24us) client_negotiation.cc:769] Sending connection context
0806 05:01:11.041909 (+ 115us) client_negotiation.cc:240] Negotiation successful
0806 05:01:11.042037 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":856,"thread_start_us":1252,"threads_started":1}
I20260806 05:01:11.043236 1391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:01:11.034279 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.96.66:41859 (local address 127.9.96.67:41539)
0806 05:01:11.034520 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0806 05:01:11.034528 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 05:01:11.035700 (+ 1172us) server_negotiation.cc:407] Connection header received
0806 05:01:11.035836 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 05:01:11.035843 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 05:01:11.035939 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 05:01:11.036054 (+ 115us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 05:01:11.036723 (+ 669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:01:11.037512 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:01:11.040119 (+ 2607us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:01:11.041803 (+ 1684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:01:11.041962 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:01:11.042100 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0806 05:01:11.042794 (+ 694us) server_negotiation.cc:299] Negotiation successful
0806 05:01:11.042975 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260806 05:01:11.043411 1403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:01:11.034108 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.67:41539 (local address 127.9.96.66:41859)
0806 05:01:11.035584 (+ 1476us) negotiation.cc:107] Waiting for socket to connect
0806 05:01:11.035602 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 05:01:11.035717 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 05:01:11.036084 (+ 367us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 05:01:11.036093 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 05:01:11.036111 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 05:01:11.036561 (+ 450us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 05:01:11.036570 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 05:01:11.037673 (+ 1103us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 05:01:11.037679 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 05:01:11.039939 (+ 2260us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 05:01:11.039954 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 05:01:11.042286 (+ 2332us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 05:01:11.042296 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 05:01:11.042567 (+ 271us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:01:11.042619 (+ 52us) client_negotiation.cc:769] Sending connection context
0806 05:01:11.042741 (+ 122us) client_negotiation.cc:240] Negotiation successful
0806 05:01:11.042948 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1360,"thread_start_us":751,"threads_started":1}
I20260806 05:01:11.043751 885 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c2e4ae9e26b142abb7c57c69dd511f51" candidate_uuid: "2c79466ec5a14051bbc556f5a3852645" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "14be68f793794acaba3556a8e1ef3c13" is_pre_election: true
I20260806 05:01:11.044343 885 raft_consensus.cc:2454] T c2e4ae9e26b142abb7c57c69dd511f51 P 14be68f793794acaba3556a8e1ef3c13 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c79466ec5a14051bbc556f5a3852645 in term 0.
I20260806 05:01:11.044528 1298 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c2e4ae9e26b142abb7c57c69dd511f51" candidate_uuid: "2c79466ec5a14051bbc556f5a3852645" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ebbb98128aa4bd69728676853db94a2" is_pre_election: true
I20260806 05:01:11.045089 990 leader_election.cc:302] T c2e4ae9e26b142abb7c57c69dd511f51 P 2c79466ec5a14051bbc556f5a3852645 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 14be68f793794acaba3556a8e1ef3c13, 2c79466ec5a14051bbc556f5a3852645; no voters:
I20260806 05:01:11.045090 1298 raft_consensus.cc:2454] T c2e4ae9e26b142abb7c57c69dd511f51 P 6ebbb98128aa4bd69728676853db94a2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c79466ec5a14051bbc556f5a3852645 in term 0.
I20260806 05:01:11.045485 1399 raft_consensus.cc:2781] T c2e4ae9e26b142abb7c57c69dd511f51 P 2c79466ec5a14051bbc556f5a3852645 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 05:01:11.045644 1399 raft_consensus.cc:482] T c2e4ae9e26b142abb7c57c69dd511f51 P 2c79466ec5a14051bbc556f5a3852645 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 05:01:11.045737 1399 raft_consensus.cc:3037] T c2e4ae9e26b142abb7c57c69dd511f51 P 2c79466ec5a14051bbc556f5a3852645 [term 0 FOLLOWER]: Advancing to term 1
I20260806 05:01:11.047955 1399 raft_consensus.cc:504] T c2e4ae9e26b142abb7c57c69dd511f51 P 2c79466ec5a14051bbc556f5a3852645 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c79466ec5a14051bbc556f5a3852645" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 36483 } } peers { permanent_uuid: "14be68f793794acaba3556a8e1ef3c13" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 45547 } } peers { permanent_uuid: "6ebbb98128aa4bd69728676853db94a2" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 41539 } }
I20260806 05:01:11.048559 1399 leader_election.cc:288] T c2e4ae9e26b142abb7c57c69dd511f51 P 2c79466ec5a14051bbc556f5a3852645 [CANDIDATE]: Term 1 election: Requested vote from peers 14be68f793794acaba3556a8e1ef3c13 (127.9.96.65:45547), 6ebbb98128aa4bd69728676853db94a2 (127.9.96.67:41539)
I20260806 05:01:11.048939 885 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c2e4ae9e26b142abb7c57c69dd511f51" candidate_uuid: "2c79466ec5a14051bbc556f5a3852645" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "14be68f793794acaba3556a8e1ef3c13"
I20260806 05:01:11.049052 1298 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c2e4ae9e26b142abb7c57c69dd511f51" candidate_uuid: "2c79466ec5a14051bbc556f5a3852645" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ebbb98128aa4bd69728676853db94a2"
I20260806 05:01:11.049175 885 raft_consensus.cc:3037] T c2e4ae9e26b142abb7c57c69dd511f51 P 14be68f793794acaba3556a8e1ef3c13 [term 0 FOLLOWER]: Advancing to term 1
I20260806 05:01:11.049285 1298 raft_consensus.cc:3037] T c2e4ae9e26b142abb7c57c69dd511f51 P 6ebbb98128aa4bd69728676853db94a2 [term 0 FOLLOWER]: Advancing to term 1
I20260806 05:01:11.052145 1298 raft_consensus.cc:2454] T c2e4ae9e26b142abb7c57c69dd511f51 P 6ebbb98128aa4bd69728676853db94a2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c79466ec5a14051bbc556f5a3852645 in term 1.
I20260806 05:01:11.052558 988 leader_election.cc:302] T c2e4ae9e26b142abb7c57c69dd511f51 P 2c79466ec5a14051bbc556f5a3852645 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c79466ec5a14051bbc556f5a3852645, 6ebbb98128aa4bd69728676853db94a2; no voters:
I20260806 05:01:11.052873 1399 raft_consensus.cc:2781] T c2e4ae9e26b142abb7c57c69dd511f51 P 2c79466ec5a14051bbc556f5a3852645 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 05:01:11.053139 885 raft_consensus.cc:2454] T c2e4ae9e26b142abb7c57c69dd511f51 P 14be68f793794acaba3556a8e1ef3c13 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c79466ec5a14051bbc556f5a3852645 in term 1.
I20260806 05:01:11.053507 1399 raft_consensus.cc:686] T c2e4ae9e26b142abb7c57c69dd511f51 P 2c79466ec5a14051bbc556f5a3852645 [term 1 LEADER]: Becoming Leader. State: Replica: 2c79466ec5a14051bbc556f5a3852645, State: Running, Role: LEADER
I20260806 05:01:11.054085 1399 consensus_queue.cc:237] T c2e4ae9e26b142abb7c57c69dd511f51 P 2c79466ec5a14051bbc556f5a3852645 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c79466ec5a14051bbc556f5a3852645" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 36483 } } peers { permanent_uuid: "14be68f793794acaba3556a8e1ef3c13" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 45547 } } peers { permanent_uuid: "6ebbb98128aa4bd69728676853db94a2" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 41539 } }
I20260806 05:01:11.058637 668 catalog_manager.cc:5495] T c2e4ae9e26b142abb7c57c69dd511f51 P 2c79466ec5a14051bbc556f5a3852645 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2c79466ec5a14051bbc556f5a3852645 (127.9.96.66). New cstate: current_term: 1 leader_uuid: "2c79466ec5a14051bbc556f5a3852645" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c79466ec5a14051bbc556f5a3852645" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 36483 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "14be68f793794acaba3556a8e1ef3c13" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 45547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6ebbb98128aa4bd69728676853db94a2" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 41539 } health_report { overall_health: UNKNOWN } } }
I20260806 05:01:11.083081 1385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:01:11.076170 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43706 (local address 127.9.96.66:36483)
0806 05:01:11.076407 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0806 05:01:11.076415 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 05:01:11.076439 (+ 24us) server_negotiation.cc:407] Connection header received
0806 05:01:11.076556 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 05:01:11.076564 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 05:01:11.076691 (+ 127us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 05:01:11.076875 (+ 184us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 05:01:11.077204 (+ 329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:01:11.078602 (+ 1398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:01:11.080114 (+ 1512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:01:11.080915 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:01:11.081064 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:01:11.081575 (+ 511us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 05:01:11.082387 (+ 812us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 05:01:11.082550 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0806 05:01:11.082606 (+ 56us) server_negotiation.cc:299] Negotiation successful
0806 05:01:11.082707 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260806 05:01:11.092872 885 raft_consensus.cc:1261] T c2e4ae9e26b142abb7c57c69dd511f51 P 14be68f793794acaba3556a8e1ef3c13 [term 1 FOLLOWER]: Refusing update from remote peer 2c79466ec5a14051bbc556f5a3852645: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 05:01:11.093119 1298 raft_consensus.cc:1261] T c2e4ae9e26b142abb7c57c69dd511f51 P 6ebbb98128aa4bd69728676853db94a2 [term 1 FOLLOWER]: Refusing update from remote peer 2c79466ec5a14051bbc556f5a3852645: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 05:01:11.093948 1405 consensus_queue.cc:1035] T c2e4ae9e26b142abb7c57c69dd511f51 P 2c79466ec5a14051bbc556f5a3852645 [LEADER]: Connected to new peer: Peer: permanent_uuid: "14be68f793794acaba3556a8e1ef3c13" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 45547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 05:01:11.094349 1399 consensus_queue.cc:1035] T c2e4ae9e26b142abb7c57c69dd511f51 P 2c79466ec5a14051bbc556f5a3852645 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ebbb98128aa4bd69728676853db94a2" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 41539 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 05:01:11.115069 1414 mvcc.cc:205] Tried to move back new op lower bound from 7315425161586593792 to 7315425161441296384. Current Snapshot: MvccSnapshot[applied={T|T < 7315425161586593792}]
I20260806 05:01:11.134793 668 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43684:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 05:01:11.135210 668 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43684:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 05:01:11.142479 668 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 28af3f4cbb8141709b39138ff8f369b3: Sending DeleteTablet for 3 replicas of tablet c2e4ae9e26b142abb7c57c69dd511f51
I20260806 05:01:11.144062 9601 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu pid 766
I20260806 05:01:11.144340 854 tablet_service.cc:1514] Processing DeleteTablet for tablet c2e4ae9e26b142abb7c57c69dd511f51 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 05:01:11 UTC) from {username='kudu', principal='kudu/127.9.96.126@KRBTEST.COM'} at 127.0.0.1:56390
I20260806 05:01:11.144351 1057 tablet_service.cc:1514] Processing DeleteTablet for tablet c2e4ae9e26b142abb7c57c69dd511f51 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 05:01:11 UTC) from {username='kudu', principal='kudu/127.9.96.126@KRBTEST.COM'} at 127.0.0.1:43694
I20260806 05:01:11.145011 853 tablet_service.cc:1514] Processing DeleteTablet for tablet c2e4ae9e26b142abb7c57c69dd511f51 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 05:01:11 UTC) from {username='kudu', principal='kudu/127.9.96.126@KRBTEST.COM'} at 127.0.0.1:56390
I20260806 05:01:11.145654 1266 tablet_service.cc:1514] Processing DeleteTablet for tablet c2e4ae9e26b142abb7c57c69dd511f51 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 05:01:11 UTC) from {username='kudu', principal='kudu/127.9.96.126@KRBTEST.COM'} at 127.0.0.1:57950
I20260806 05:01:11.154093 1424 tablet_replica.cc:330] T c2e4ae9e26b142abb7c57c69dd511f51 P 14be68f793794acaba3556a8e1ef3c13: stopping tablet replica
I20260806 05:01:11.154667 1424 raft_consensus.cc:2229] T c2e4ae9e26b142abb7c57c69dd511f51 P 14be68f793794acaba3556a8e1ef3c13 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 05:01:11.155359 1424 raft_consensus.cc:2258] T c2e4ae9e26b142abb7c57c69dd511f51 P 14be68f793794acaba3556a8e1ef3c13 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 05:01:11.159185 1428 tablet_replica.cc:330] T c2e4ae9e26b142abb7c57c69dd511f51 P 6ebbb98128aa4bd69728676853db94a2: stopping tablet replica
I20260806 05:01:11.159724 1425 tablet_replica.cc:330] T c2e4ae9e26b142abb7c57c69dd511f51 P 2c79466ec5a14051bbc556f5a3852645: stopping tablet replica
I20260806 05:01:11.159849 1428 raft_consensus.cc:2229] T c2e4ae9e26b142abb7c57c69dd511f51 P 6ebbb98128aa4bd69728676853db94a2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 05:01:11.160166 1425 raft_consensus.cc:2229] T c2e4ae9e26b142abb7c57c69dd511f51 P 2c79466ec5a14051bbc556f5a3852645 [term 1 LEADER]: Raft consensus shutting down.
I20260806 05:01:11.160840 1425 raft_consensus.cc:2258] T c2e4ae9e26b142abb7c57c69dd511f51 P 2c79466ec5a14051bbc556f5a3852645 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 05:01:11.161175 1428 raft_consensus.cc:2258] T c2e4ae9e26b142abb7c57c69dd511f51 P 6ebbb98128aa4bd69728676853db94a2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 05:01:11.162633 1424 ts_tablet_manager.cc:1905] T c2e4ae9e26b142abb7c57c69dd511f51 P 14be68f793794acaba3556a8e1ef3c13: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 05:01:11.162901 1425 ts_tablet_manager.cc:1905] T c2e4ae9e26b142abb7c57c69dd511f51 P 2c79466ec5a14051bbc556f5a3852645: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 05:01:11.164884 1428 ts_tablet_manager.cc:1905] T c2e4ae9e26b142abb7c57c69dd511f51 P 6ebbb98128aa4bd69728676853db94a2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 05:01:11.170704 1425 ts_tablet_manager.cc:1918] T c2e4ae9e26b142abb7c57c69dd511f51 P 2c79466ec5a14051bbc556f5a3852645: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 05:01:11.170907 1425 log.cc:1196] T c2e4ae9e26b142abb7c57c69dd511f51 P 2c79466ec5a14051bbc556f5a3852645: Deleting WAL directory at /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785992227050689-9601-0/minicluster-data/ts-1/wal/wals/c2e4ae9e26b142abb7c57c69dd511f51
I20260806 05:01:11.171499 1425 ts_tablet_manager.cc:1939] T c2e4ae9e26b142abb7c57c69dd511f51 P 2c79466ec5a14051bbc556f5a3852645: Deleting consensus metadata
I20260806 05:01:11.173331 648 catalog_manager.cc:4841] TS 2c79466ec5a14051bbc556f5a3852645 (127.9.96.66:36483): tablet c2e4ae9e26b142abb7c57c69dd511f51 (table test-table [id=53f5e000f00c48d5af881d35d00ad61e]) successfully deleted
I20260806 05:01:11.179227 1424 ts_tablet_manager.cc:1918] T c2e4ae9e26b142abb7c57c69dd511f51 P 14be68f793794acaba3556a8e1ef3c13: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 05:01:11.179468 1424 log.cc:1196] T c2e4ae9e26b142abb7c57c69dd511f51 P 14be68f793794acaba3556a8e1ef3c13: Deleting WAL directory at /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785992227050689-9601-0/minicluster-data/ts-0/wal/wals/c2e4ae9e26b142abb7c57c69dd511f51
I20260806 05:01:11.179944 1424 ts_tablet_manager.cc:1939] T c2e4ae9e26b142abb7c57c69dd511f51 P 14be68f793794acaba3556a8e1ef3c13: Deleting consensus metadata
I20260806 05:01:11.181524 647 catalog_manager.cc:4841] TS 14be68f793794acaba3556a8e1ef3c13 (127.9.96.65:45547): tablet c2e4ae9e26b142abb7c57c69dd511f51 (table test-table [id=53f5e000f00c48d5af881d35d00ad61e]) successfully deleted
W20260806 05:01:11.182247 647 catalog_manager.cc:4808] TS 14be68f793794acaba3556a8e1ef3c13 (127.9.96.65:45547): delete failed for tablet c2e4ae9e26b142abb7c57c69dd511f51 because the tablet was not found. No further retry: Not found: Tablet not found: c2e4ae9e26b142abb7c57c69dd511f51
I20260806 05:01:11.185926 1428 ts_tablet_manager.cc:1918] T c2e4ae9e26b142abb7c57c69dd511f51 P 6ebbb98128aa4bd69728676853db94a2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 05:01:11.186113 1428 log.cc:1196] T c2e4ae9e26b142abb7c57c69dd511f51 P 6ebbb98128aa4bd69728676853db94a2: Deleting WAL directory at /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785992227050689-9601-0/minicluster-data/ts-2/wal/wals/c2e4ae9e26b142abb7c57c69dd511f51
I20260806 05:01:11.186661 1428 ts_tablet_manager.cc:1939] T c2e4ae9e26b142abb7c57c69dd511f51 P 6ebbb98128aa4bd69728676853db94a2: Deleting consensus metadata
I20260806 05:01:11.188606 646 catalog_manager.cc:4841] TS 6ebbb98128aa4bd69728676853db94a2 (127.9.96.67:41539): tablet c2e4ae9e26b142abb7c57c69dd511f51 (table test-table [id=53f5e000f00c48d5af881d35d00ad61e]) successfully deleted
Aug 06 05:01:11 dist-test-slave-x4qh krb5kdc[593](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992468, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.96.65@KRBTEST.COM
I20260806 05:01:11.202147 1387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:01:11.146417 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56402 (local address 127.9.96.65:45547)
0806 05:01:11.147049 (+ 632us) server_negotiation.cc:206] Beginning negotiation
0806 05:01:11.147058 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 05:01:11.171581 (+ 24523us) server_negotiation.cc:407] Connection header received
0806 05:01:11.173634 (+ 2053us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 05:01:11.173643 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 05:01:11.173730 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 05:01:11.173812 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0806 05:01:11.183529 (+ 9717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:01:11.184667 (+ 1138us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:01:11.187233 (+ 2566us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:01:11.187876 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:01:11.187989 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:01:11.194345 (+ 6356us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 05:01:11.194362 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 05:01:11.194378 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 05:01:11.194434 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0806 05:01:11.198575 (+ 4141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 05:01:11.199499 (+ 924us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 05:01:11.199508 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 05:01:11.199512 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 05:01:11.199678 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 05:01:11.200195 (+ 517us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 05:01:11.200200 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 05:01:11.200204 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 05:01:11.200638 (+ 434us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 05:01:11.200920 (+ 282us) server_negotiation.cc:1035] Waiting for connection context
0806 05:01:11.201540 (+ 620us) server_negotiation.cc:299] Negotiation successful
0806 05:01:11.201765 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":459}
I20260806 05:01:11.203608 943 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 05:01:11.312176 9601 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu with pid 766
I20260806 05:01:11.336855 9601 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu pid 960
Aug 06 05:01:11 dist-test-slave-x4qh krb5kdc[593](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992468, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.96.66@KRBTEST.COM
I20260806 05:01:11.358642 1385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:01:11.337646 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43716 (local address 127.9.96.66:36483)
0806 05:01:11.338194 (+ 548us) server_negotiation.cc:206] Beginning negotiation
0806 05:01:11.338201 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 05:01:11.338219 (+ 18us) server_negotiation.cc:407] Connection header received
0806 05:01:11.338284 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 05:01:11.338288 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 05:01:11.338358 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 05:01:11.338478 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0806 05:01:11.340193 (+ 1715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:01:11.341108 (+ 915us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:01:11.342577 (+ 1469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:01:11.343046 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:01:11.343193 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:01:11.347570 (+ 4377us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 05:01:11.347593 (+ 23us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 05:01:11.347619 (+ 26us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 05:01:11.347684 (+ 65us) server_negotiation.cc:922] calling sasl_server_start()
0806 05:01:11.353261 (+ 5577us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 05:01:11.354399 (+ 1138us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 05:01:11.354412 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 05:01:11.354418 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 05:01:11.354617 (+ 199us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 05:01:11.355202 (+ 585us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 05:01:11.355209 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 05:01:11.355215 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 05:01:11.355700 (+ 485us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 05:01:11.357521 (+ 1821us) server_negotiation.cc:1035] Waiting for connection context
0806 05:01:11.357708 (+ 187us) server_negotiation.cc:299] Negotiation successful
0806 05:01:11.357993 (+ 285us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":405}
I20260806 05:01:11.359872 1159 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 05:01:11.467525 9601 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu with pid 960
I20260806 05:01:11.494371 9601 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu pid 1174
Aug 06 05:01:11 dist-test-slave-x4qh krb5kdc[593](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992468, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.96.67@KRBTEST.COM
I20260806 05:01:11.511178 1391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:01:11.495129 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57956 (local address 127.9.96.67:41539)
0806 05:01:11.495321 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0806 05:01:11.495326 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 05:01:11.495361 (+ 35us) server_negotiation.cc:407] Connection header received
0806 05:01:11.495578 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 05:01:11.495582 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 05:01:11.495649 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 05:01:11.495742 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0806 05:01:11.497007 (+ 1265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:01:11.497725 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:01:11.498899 (+ 1174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:01:11.499291 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:01:11.499443 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:01:11.503993 (+ 4550us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 05:01:11.504006 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 05:01:11.504021 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 05:01:11.504051 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0806 05:01:11.507355 (+ 3304us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 05:01:11.508235 (+ 880us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 05:01:11.508244 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 05:01:11.508247 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 05:01:11.508396 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 05:01:11.508843 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 05:01:11.508847 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 05:01:11.508851 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 05:01:11.509273 (+ 422us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 05:01:11.509561 (+ 288us) server_negotiation.cc:1035] Waiting for connection context
0806 05:01:11.510621 (+ 1060us) server_negotiation.cc:299] Negotiation successful
0806 05:01:11.510824 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260806 05:01:11.515322 1361 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 05:01:11.604796 9601 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu with pid 1174
I20260806 05:01:11.633412 9601 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu pid 615
I20260806 05:01:11.634577 715 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 05:01:11.743626 9601 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu with pid 615
I20260806 05:01:31.775952 9601 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
/tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785992227050689-9601-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785992227050689-9601-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785992227050689-9601-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785992227050689-9601-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.96.126:34193
--webserver_interface=127.9.96.126
--webserver_port=33641
--keytab_file=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.126.keytab
--principal=kudu/127.9.96.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.96.84:41431
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.96.126:34193
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785992227050689-9601-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785992227050689-9601-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785992227050689-9601-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 05:01:32.127071 1435 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 05:01:32.127441 1435 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 05:01:32.127614 1435 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 05:01:32.134763 1435 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260806 05:01:32.134905 1435 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260806 05:01:32.135007 1435 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260806 05:01:32.135111 1435 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 05:01:32.135201 1435 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 05:01:32.135308 1435 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 05:01:32.135426 1435 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 05:01:32.135511 1435 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 05:01:32.145839 1435 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.96.84:41431
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785992227050689-9601-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785992227050689-9601-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.9.96.126:34193
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.126.keytab
--principal=kudu/127.9.96.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.96.126:34193
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785992227050689-9601-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.96.126
--webserver_port=33641
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785992227050689-9601-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 03:56:12 UTC on ccycloud.quasar-guijbm.root.comops.site
build id 4569618
ASAN enabled
I20260806 05:01:32.147203 1435 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 05:01:32.148591 1435 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 05:01:32.174218 1444 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 05:01:32.174223 1445 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 05:01:32.174212 1448 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 05:01:32.175087 1435 server_base.cc:1034] running on GCE node
I20260806 05:01:32.176036 1435 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 05:01:32.177541 1435 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 05:01:32.178834 1435 hybrid_clock.cc:648] HybridClock initialized: now 1785992492178746 us; error 58 us; skew 500 ppm
I20260806 05:01:32.179234 1435 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 05:01:32 dist-test-slave-x4qh krb5kdc[593](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992492, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.96.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 05:01:32.187350 1435 init.cc:377] Logged in from keytab as kudu/127.9.96.126@KRBTEST.COM (short username kudu)
I20260806 05:01:32.189218 1435 webserver.cc:460] Webserver started at http://127.9.96.126:33641/ using document root <none> and password file <none>
I20260806 05:01:32.190060 1435 fs_manager.cc:359] Metadata directory not provided
I20260806 05:01:32.190224 1435 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 05:01:32.196563 1435 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.002s sys 0.001s
I20260806 05:01:32.199568 1458 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 05:01:32.200891 1435 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260806 05:01:32.201086 1435 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785992227050689-9601-0/minicluster-data/master-0/data,/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785992227050689-9601-0/minicluster-data/master-0/wal
uuid: "28af3f4cbb8141709b39138ff8f369b3"
format_stamp: "Formatted at 2026-08-06 05:01:08 on dist-test-slave-x4qh"
server_key: "c36b42d5d25059f8032ec59f36e1c6cf"
server_key_iv: "97e75fb038a44ae429b93be19508ec27"
server_key_version: "encryptionkey@0"
I20260806 05:01:32.202145 1435 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785992227050689-9601-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785992227050689-9601-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785992227050689-9601-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 05:01:32.216915 1435 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 05:01:32.220944 1435 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 05:01:32.221261 1435 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 05:01:32.240904 1435 rpc_server.cc:307] RPC server started. Bound to: 127.9.96.126:34193
I20260806 05:01:32.240958 1533 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.96.126:34193 every 8 connection(s)
I20260806 05:01:32.242457 1435 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785992227050689-9601-0/minicluster-data/master-0/data/info.pb
I20260806 05:01:32.247495 9601 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu as pid 1435
I20260806 05:01:32.248584 9601 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
/tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785992227050689-9601-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785992227050689-9601-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785992227050689-9601-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785992227050689-9601-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.96.65:45547
--local_ip_for_outbound_sockets=127.9.96.65
--tserver_master_addrs=127.9.96.126:34193
--webserver_port=44495
--webserver_interface=127.9.96.65
--keytab_file=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.65.keytab
--principal=kudu/127.9.96.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.96.84:41431
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785992227050689-9601-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785992227050689-9601-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785992227050689-9601-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 05:01:32.250926 1534 sys_catalog.cc:263] Verifying existing consensus state
I20260806 05:01:32.256295 1534 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 28af3f4cbb8141709b39138ff8f369b3: Bootstrap starting.
I20260806 05:01:32.277904 1534 log.cc:824] T 00000000000000000000000000000000 P 28af3f4cbb8141709b39138ff8f369b3: Log is configured to *not* fsync() on all Append() calls
I20260806 05:01:32.290685 1534 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 28af3f4cbb8141709b39138ff8f369b3: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 05:01:32.291266 1534 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 28af3f4cbb8141709b39138ff8f369b3: Bootstrap complete.
I20260806 05:01:32.300781 1534 raft_consensus.cc:348] T 00000000000000000000000000000000 P 28af3f4cbb8141709b39138ff8f369b3 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "28af3f4cbb8141709b39138ff8f369b3" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 34193 } }
I20260806 05:01:32.301677 1534 raft_consensus.cc:729] T 00000000000000000000000000000000 P 28af3f4cbb8141709b39138ff8f369b3 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 28af3f4cbb8141709b39138ff8f369b3, State: Initialized, Role: FOLLOWER
I20260806 05:01:32.302342 1534 consensus_queue.cc:260] T 00000000000000000000000000000000 P 28af3f4cbb8141709b39138ff8f369b3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "28af3f4cbb8141709b39138ff8f369b3" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 34193 } }
I20260806 05:01:32.302627 1534 raft_consensus.cc:388] T 00000000000000000000000000000000 P 28af3f4cbb8141709b39138ff8f369b3 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 05:01:32.302760 1534 raft_consensus.cc:482] T 00000000000000000000000000000000 P 28af3f4cbb8141709b39138ff8f369b3 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 05:01:32.302956 1534 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 28af3f4cbb8141709b39138ff8f369b3 [term 1 FOLLOWER]: Advancing to term 2
I20260806 05:01:32.307341 1534 raft_consensus.cc:504] T 00000000000000000000000000000000 P 28af3f4cbb8141709b39138ff8f369b3 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "28af3f4cbb8141709b39138ff8f369b3" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 34193 } }
I20260806 05:01:32.307981 1534 leader_election.cc:302] T 00000000000000000000000000000000 P 28af3f4cbb8141709b39138ff8f369b3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 28af3f4cbb8141709b39138ff8f369b3; no voters:
I20260806 05:01:32.308689 1534 leader_election.cc:288] T 00000000000000000000000000000000 P 28af3f4cbb8141709b39138ff8f369b3 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260806 05:01:32.308843 1540 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 28af3f4cbb8141709b39138ff8f369b3 [term 2 FOLLOWER]: Leader election won for term 2
I20260806 05:01:32.311331 1534 sys_catalog.cc:564] T 00000000000000000000000000000000 P 28af3f4cbb8141709b39138ff8f369b3 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 05:01:32.315896 1540 raft_consensus.cc:686] T 00000000000000000000000000000000 P 28af3f4cbb8141709b39138ff8f369b3 [term 2 LEADER]: Becoming Leader. State: Replica: 28af3f4cbb8141709b39138ff8f369b3, State: Running, Role: LEADER
I20260806 05:01:32.316632 1540 consensus_queue.cc:237] T 00000000000000000000000000000000 P 28af3f4cbb8141709b39138ff8f369b3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "28af3f4cbb8141709b39138ff8f369b3" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 34193 } }
I20260806 05:01:32.319928 1541 sys_catalog.cc:455] T 00000000000000000000000000000000 P 28af3f4cbb8141709b39138ff8f369b3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "28af3f4cbb8141709b39138ff8f369b3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "28af3f4cbb8141709b39138ff8f369b3" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 34193 } } }
I20260806 05:01:32.320536 1541 sys_catalog.cc:458] T 00000000000000000000000000000000 P 28af3f4cbb8141709b39138ff8f369b3 [sys.catalog]: This master's current role is: LEADER
I20260806 05:01:32.321149 1542 sys_catalog.cc:455] T 00000000000000000000000000000000 P 28af3f4cbb8141709b39138ff8f369b3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 28af3f4cbb8141709b39138ff8f369b3. Latest consensus state: current_term: 2 leader_uuid: "28af3f4cbb8141709b39138ff8f369b3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "28af3f4cbb8141709b39138ff8f369b3" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 34193 } } }
I20260806 05:01:32.321516 1542 sys_catalog.cc:458] T 00000000000000000000000000000000 P 28af3f4cbb8141709b39138ff8f369b3 [sys.catalog]: This master's current role is: LEADER
I20260806 05:01:32.330681 1554 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 05:01:32.341485 1554 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 05:01:32.343477 1554 catalog_manager.cc:1259] Loaded cluster ID: d13035ae4ef14dfab1dc54b0a8fbf05e
I20260806 05:01:32.344187 1554 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 05:01:32.347594 1554 catalog_manager.cc:1504] Loading token signing keys...
I20260806 05:01:32.348799 1554 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 28af3f4cbb8141709b39138ff8f369b3: Loaded TSK: 0
I20260806 05:01:32.377373 1554 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 28af3f4cbb8141709b39138ff8f369b3: Generated new TSK 1
I20260806 05:01:32.377738 1554 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260806 05:01:32.632436 1538 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 05:01:32.632689 1538 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 05:01:32.632833 1538 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 05:01:32.640203 1538 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 05:01:32.640312 1538 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 05:01:32.640482 1538 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.96.65
I20260806 05:01:32.650556 1538 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.96.84:41431
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785992227050689-9601-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785992227050689-9601-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.65.keytab
--principal=kudu/127.9.96.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.96.65:45547
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785992227050689-9601-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.96.65
--webserver_port=44495
--tserver_master_addrs=127.9.96.126:34193
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.96.65
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785992227050689-9601-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 03:56:12 UTC on ccycloud.quasar-guijbm.root.comops.site
build id 4569618
ASAN enabled
I20260806 05:01:32.651935 1538 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 05:01:32.653276 1538 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 05:01:32.668390 1573 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 05:01:32.668947 1572 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 05:01:32.669495 1575 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 05:01:32.669799 1538 server_base.cc:1034] running on GCE node
I20260806 05:01:32.670501 1538 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 05:01:32.671594 1538 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 05:01:32.672849 1538 hybrid_clock.cc:648] HybridClock initialized: now 1785992492672780 us; error 54 us; skew 500 ppm
I20260806 05:01:32.673187 1538 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 05:01:32 dist-test-slave-x4qh krb5kdc[593](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992492, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.96.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 05:01:32.676592 1538 init.cc:377] Logged in from keytab as kudu/127.9.96.65@KRBTEST.COM (short username kudu)
I20260806 05:01:32.678287 1538 webserver.cc:460] Webserver started at http://127.9.96.65:44495/ using document root <none> and password file <none>
I20260806 05:01:32.679006 1538 fs_manager.cc:359] Metadata directory not provided
I20260806 05:01:32.679175 1538 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 05:01:32.684777 1538 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.000s sys 0.004s
I20260806 05:01:32.687927 1586 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 05:01:32.689332 1538 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260806 05:01:32.689509 1538 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785992227050689-9601-0/minicluster-data/ts-0/data,/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785992227050689-9601-0/minicluster-data/ts-0/wal
uuid: "14be68f793794acaba3556a8e1ef3c13"
format_stamp: "Formatted at 2026-08-06 05:01:09 on dist-test-slave-x4qh"
server_key: "3128738e1e7ef5228a1daa36f098d2f4"
server_key_iv: "7922f313fa41aa73b12b8f5bc3e10018"
server_key_version: "encryptionkey@0"
I20260806 05:01:32.690477 1538 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785992227050689-9601-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785992227050689-9601-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785992227050689-9601-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 05:01:32.707113 1538 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 05:01:32.711571 1538 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 05:01:32.711848 1538 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 05:01:32.712942 1538 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 05:01:32.714272 1538 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 05:01:32.714418 1538 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 05:01:32.714562 1538 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 05:01:32.714668 1538 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 05:01:32.752059 1538 rpc_server.cc:307] RPC server started. Bound to: 127.9.96.65:45547
I20260806 05:01:32.752152 1739 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.96.65:45547 every 8 connection(s)
I20260806 05:01:32.753553 1538 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785992227050689-9601-0/minicluster-data/ts-0/data/info.pb
I20260806 05:01:32.760239 9601 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu as pid 1538
I20260806 05:01:32.761111 9601 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
/tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785992227050689-9601-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785992227050689-9601-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785992227050689-9601-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785992227050689-9601-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.96.66:36483
--local_ip_for_outbound_sockets=127.9.96.66
--tserver_master_addrs=127.9.96.126:34193
--webserver_port=37521
--webserver_interface=127.9.96.66
--keytab_file=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.66.keytab
--principal=kudu/127.9.96.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.96.84:41431
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785992227050689-9601-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785992227050689-9601-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785992227050689-9601-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 05:01:32 dist-test-slave-x4qh krb5kdc[593](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992492, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.96.65@KRBTEST.COM for kudu/127.9.96.126@KRBTEST.COM
I20260806 05:01:32.778914 1744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:01:32.756559 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.96.65:33365 (local address 127.9.96.126:34193)
0806 05:01:32.757215 (+ 656us) server_negotiation.cc:206] Beginning negotiation
0806 05:01:32.757226 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 05:01:32.757696 (+ 470us) server_negotiation.cc:407] Connection header received
0806 05:01:32.760023 (+ 2327us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 05:01:32.760043 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 05:01:32.760496 (+ 453us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 05:01:32.760928 (+ 432us) server_negotiation.cc:226] Negotiated authn=SASL
0806 05:01:32.762418 (+ 1490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:01:32.764514 (+ 2096us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:01:32.766396 (+ 1882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:01:32.767555 (+ 1159us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:01:32.767757 (+ 202us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:01:32.771330 (+ 3573us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 05:01:32.771343 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 05:01:32.771365 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 05:01:32.771443 (+ 78us) server_negotiation.cc:922] calling sasl_server_start()
0806 05:01:32.774489 (+ 3046us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 05:01:32.775483 (+ 994us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 05:01:32.775489 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 05:01:32.775494 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 05:01:32.775658 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 05:01:32.776186 (+ 528us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 05:01:32.776189 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 05:01:32.776191 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 05:01:32.776618 (+ 427us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 05:01:32.776773 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0806 05:01:32.777709 (+ 936us) server_negotiation.cc:299] Negotiation successful
0806 05:01:32.778019 (+ 310us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":461,"thread_start_us":322,"threads_started":1}
I20260806 05:01:32.779482 1745 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:01:32.756638 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:34193 (local address 127.9.96.65:33365)
0806 05:01:32.757495 (+ 857us) negotiation.cc:107] Waiting for socket to connect
0806 05:01:32.757551 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0806 05:01:32.759317 (+ 1766us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 05:01:32.761013 (+ 1696us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 05:01:32.761020 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 05:01:32.761649 (+ 629us) client_negotiation.cc:189] Negotiated authn=SASL
0806 05:01:32.762170 (+ 521us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 05:01:32.762188 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 05:01:32.764706 (+ 2518us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 05:01:32.764714 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 05:01:32.766236 (+ 1522us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 05:01:32.766249 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 05:01:32.767765 (+ 1516us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 05:01:32.767770 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 05:01:32.767950 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:01:32.768652 (+ 702us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 05:01:32.768673 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0806 05:01:32.771037 (+ 2364us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 05:01:32.774646 (+ 3609us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 05:01:32.774658 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 05:01:32.774675 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0806 05:01:32.775317 (+ 642us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 05:01:32.775805 (+ 488us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 05:01:32.775812 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 05:01:32.775827 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0806 05:01:32.776075 (+ 248us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 05:01:32.776782 (+ 707us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 05:01:32.776792 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 05:01:32.777245 (+ 453us) client_negotiation.cc:769] Sending connection context
0806 05:01:32.777551 (+ 306us) client_negotiation.cc:240] Negotiation successful
0806 05:01:32.778124 (+ 573us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":565,"thread_start_us":381,"threads_started":1}
I20260806 05:01:32.782241 1742 heartbeater.cc:347] Connected to a master server at 127.9.96.126:34193
I20260806 05:01:32.782559 1742 heartbeater.cc:464] Registering TS with master...
I20260806 05:01:32.783277 1742 heartbeater.cc:511] Master 127.9.96.126:34193 requested a full tablet report, sending...
I20260806 05:01:32.785601 1486 ts_manager.cc:194] Registered new tserver with Master: 14be68f793794acaba3556a8e1ef3c13 (127.9.96.65:45547)
I20260806 05:01:32.787035 1486 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.96.65@KRBTEST.COM'} at 127.9.96.65:33365
W20260806 05:01:33.106490 1746 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 05:01:33.106833 1746 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 05:01:33.107019 1746 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 05:01:33.114575 1746 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 05:01:33.114678 1746 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 05:01:33.114881 1746 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.96.66
I20260806 05:01:33.124732 1746 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.96.84:41431
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785992227050689-9601-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785992227050689-9601-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.66.keytab
--principal=kudu/127.9.96.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.96.66:36483
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785992227050689-9601-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.96.66
--webserver_port=37521
--tserver_master_addrs=127.9.96.126:34193
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.96.66
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785992227050689-9601-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 03:56:12 UTC on ccycloud.quasar-guijbm.root.comops.site
build id 4569618
ASAN enabled
I20260806 05:01:33.126016 1746 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 05:01:33.127300 1746 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 05:01:33.141048 1756 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 05:01:33.141219 1755 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 05:01:33.141224 1758 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 05:01:33.142184 1746 server_base.cc:1034] running on GCE node
I20260806 05:01:33.142833 1746 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 05:01:33.144052 1746 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 05:01:33.145326 1746 hybrid_clock.cc:648] HybridClock initialized: now 1785992493145264 us; error 35 us; skew 500 ppm
I20260806 05:01:33.145711 1746 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 05:01:33 dist-test-slave-x4qh krb5kdc[593](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992493, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.96.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 05:01:33.149106 1746 init.cc:377] Logged in from keytab as kudu/127.9.96.66@KRBTEST.COM (short username kudu)
I20260806 05:01:33.150671 1746 webserver.cc:460] Webserver started at http://127.9.96.66:37521/ using document root <none> and password file <none>
I20260806 05:01:33.151505 1746 fs_manager.cc:359] Metadata directory not provided
I20260806 05:01:33.151692 1746 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 05:01:33.157269 1746 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260806 05:01:33.160080 1773 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 05:01:33.161346 1746 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 05:01:33.161530 1746 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785992227050689-9601-0/minicluster-data/ts-1/data,/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785992227050689-9601-0/minicluster-data/ts-1/wal
uuid: "2c79466ec5a14051bbc556f5a3852645"
format_stamp: "Formatted at 2026-08-06 05:01:10 on dist-test-slave-x4qh"
server_key: "9e64761b3a121e6ad69b6aa80e61efae"
server_key_iv: "7a1c3766778b4209965db7e3fa55f2ef"
server_key_version: "encryptionkey@0"
I20260806 05:01:33.162499 1746 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785992227050689-9601-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785992227050689-9601-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785992227050689-9601-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 05:01:33.184160 1746 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 05:01:33.188282 1746 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 05:01:33.188576 1746 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 05:01:33.189548 1746 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 05:01:33.190938 1746 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 05:01:33.191028 1746 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 05:01:33.191177 1746 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 05:01:33.191267 1746 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 05:01:33.226411 1746 rpc_server.cc:307] RPC server started. Bound to: 127.9.96.66:36483
I20260806 05:01:33.226495 1943 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.96.66:36483 every 8 connection(s)
I20260806 05:01:33.228106 1746 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785992227050689-9601-0/minicluster-data/ts-1/data/info.pb
I20260806 05:01:33.233219 9601 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu as pid 1746
I20260806 05:01:33.233989 9601 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
/tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785992227050689-9601-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785992227050689-9601-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785992227050689-9601-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785992227050689-9601-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.96.67:41539
--local_ip_for_outbound_sockets=127.9.96.67
--tserver_master_addrs=127.9.96.126:34193
--webserver_port=33175
--webserver_interface=127.9.96.67
--keytab_file=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.67.keytab
--principal=kudu/127.9.96.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.96.84:41431
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785992227050689-9601-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785992227050689-9601-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785992227050689-9601-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 05:01:33 dist-test-slave-x4qh krb5kdc[593](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992493, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.96.66@KRBTEST.COM for kudu/127.9.96.126@KRBTEST.COM
I20260806 05:01:33.252712 1744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:01:33.230683 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.96.66:56215 (local address 127.9.96.126:34193)
0806 05:01:33.230855 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0806 05:01:33.230860 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 05:01:33.232109 (+ 1249us) server_negotiation.cc:407] Connection header received
0806 05:01:33.234625 (+ 2516us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 05:01:33.234630 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 05:01:33.234712 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 05:01:33.234810 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0806 05:01:33.236882 (+ 2072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:01:33.237635 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:01:33.240149 (+ 2514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:01:33.240993 (+ 844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:01:33.241109 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:01:33.246157 (+ 5048us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 05:01:33.246165 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 05:01:33.246171 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 05:01:33.246219 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0806 05:01:33.249671 (+ 3452us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 05:01:33.250388 (+ 717us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 05:01:33.250395 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 05:01:33.250403 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0806 05:01:33.250516 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 05:01:33.250955 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 05:01:33.250960 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 05:01:33.250965 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 05:01:33.251240 (+ 275us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 05:01:33.251439 (+ 199us) server_negotiation.cc:1035] Waiting for connection context
0806 05:01:33.252099 (+ 660us) server_negotiation.cc:299] Negotiation successful
0806 05:01:33.252354 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260806 05:01:33.253697 1947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:01:33.231123 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:34193 (local address 127.9.96.66:56215)
0806 05:01:33.231933 (+ 810us) negotiation.cc:107] Waiting for socket to connect
0806 05:01:33.231991 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0806 05:01:33.234330 (+ 2339us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 05:01:33.235077 (+ 747us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 05:01:33.235090 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 05:01:33.235904 (+ 814us) client_negotiation.cc:189] Negotiated authn=SASL
0806 05:01:33.236687 (+ 783us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 05:01:33.236712 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 05:01:33.237804 (+ 1092us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 05:01:33.237811 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 05:01:33.239959 (+ 2148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 05:01:33.239976 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 05:01:33.241498 (+ 1522us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 05:01:33.241506 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 05:01:33.241878 (+ 372us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:01:33.242624 (+ 746us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 05:01:33.242666 (+ 42us) client_negotiation.cc:656] calling sasl_client_start()
0806 05:01:33.245845 (+ 3179us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 05:01:33.249824 (+ 3979us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 05:01:33.249833 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 05:01:33.249847 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0806 05:01:33.250252 (+ 405us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 05:01:33.250630 (+ 378us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 05:01:33.250634 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 05:01:33.250638 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 05:01:33.250833 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 05:01:33.251402 (+ 569us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 05:01:33.251410 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 05:01:33.251738 (+ 328us) client_negotiation.cc:769] Sending connection context
0806 05:01:33.252004 (+ 266us) client_negotiation.cc:240] Negotiation successful
0806 05:01:33.252506 (+ 502us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":564,"thread_start_us":364,"threads_started":1}
I20260806 05:01:33.256729 1944 heartbeater.cc:347] Connected to a master server at 127.9.96.126:34193
I20260806 05:01:33.257085 1944 heartbeater.cc:464] Registering TS with master...
I20260806 05:01:33.257844 1944 heartbeater.cc:511] Master 127.9.96.126:34193 requested a full tablet report, sending...
I20260806 05:01:33.259272 1486 ts_manager.cc:194] Registered new tserver with Master: 2c79466ec5a14051bbc556f5a3852645 (127.9.96.66:36483)
I20260806 05:01:33.260178 1486 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.96.66@KRBTEST.COM'} at 127.9.96.66:56215
W20260806 05:01:33.586385 1948 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 05:01:33.586658 1948 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 05:01:33.586751 1948 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 05:01:33.593989 1948 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 05:01:33.594092 1948 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 05:01:33.594211 1948 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.96.67
I20260806 05:01:33.603686 1948 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.96.84:41431
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785992227050689-9601-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785992227050689-9601-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.67.keytab
--principal=kudu/127.9.96.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.96.67:41539
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785992227050689-9601-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.96.67
--webserver_port=33175
--tserver_master_addrs=127.9.96.126:34193
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.96.67
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785992227050689-9601-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 03:56:12 UTC on ccycloud.quasar-guijbm.root.comops.site
build id 4569618
ASAN enabled
I20260806 05:01:33.604893 1948 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 05:01:33.606225 1948 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 05:01:33.619550 1955 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 05:01:33.619517 1960 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 05:01:33.619633 1957 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 05:01:33.620625 1948 server_base.cc:1034] running on GCE node
I20260806 05:01:33.621295 1948 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 05:01:33.622208 1948 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 05:01:33.623492 1948 hybrid_clock.cc:648] HybridClock initialized: now 1785992493623446 us; error 55 us; skew 500 ppm
I20260806 05:01:33.623821 1948 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 05:01:33 dist-test-slave-x4qh krb5kdc[593](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992493, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.96.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 05:01:33.626881 1948 init.cc:377] Logged in from keytab as kudu/127.9.96.67@KRBTEST.COM (short username kudu)
I20260806 05:01:33.628360 1948 webserver.cc:460] Webserver started at http://127.9.96.67:33175/ using document root <none> and password file <none>
I20260806 05:01:33.629024 1948 fs_manager.cc:359] Metadata directory not provided
I20260806 05:01:33.629189 1948 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 05:01:33.634279 1948 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260806 05:01:33.636861 1968 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 05:01:33.638132 1948 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260806 05:01:33.638319 1948 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785992227050689-9601-0/minicluster-data/ts-2/data,/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785992227050689-9601-0/minicluster-data/ts-2/wal
uuid: "6ebbb98128aa4bd69728676853db94a2"
format_stamp: "Formatted at 2026-08-06 05:01:10 on dist-test-slave-x4qh"
server_key: "696c825852d11aca183a8bfa31ff49da"
server_key_iv: "008ddaa8c2b7b319919f9c88a012add6"
server_key_version: "encryptionkey@0"
I20260806 05:01:33.639286 1948 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785992227050689-9601-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785992227050689-9601-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785992227050689-9601-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 05:01:33.661960 1948 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 05:01:33.665799 1948 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 05:01:33.666110 1948 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 05:01:33.667086 1948 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 05:01:33.668450 1948 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 05:01:33.668594 1948 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 05:01:33.668718 1948 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 05:01:33.668816 1948 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 05:01:33.703063 1948 rpc_server.cc:307] RPC server started. Bound to: 127.9.96.67:41539
I20260806 05:01:33.703146 2128 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.96.67:41539 every 8 connection(s)
I20260806 05:01:33.704530 1948 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785992227050689-9601-0/minicluster-data/ts-2/data/info.pb
I20260806 05:01:33.710961 9601 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu as pid 1948
Aug 06 05:01:33 dist-test-slave-x4qh krb5kdc[593](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992493, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.96.67@KRBTEST.COM for kudu/127.9.96.126@KRBTEST.COM
I20260806 05:01:33.726279 1744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:01:33.707201 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.96.67:48865 (local address 127.9.96.126:34193)
0806 05:01:33.707416 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0806 05:01:33.707422 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 05:01:33.708561 (+ 1139us) server_negotiation.cc:407] Connection header received
0806 05:01:33.710380 (+ 1819us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 05:01:33.710385 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 05:01:33.710479 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 05:01:33.710590 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0806 05:01:33.712613 (+ 2023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:01:33.713710 (+ 1097us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:01:33.715431 (+ 1721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:01:33.715883 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:01:33.716023 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:01:33.719427 (+ 3404us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 05:01:33.719435 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 05:01:33.719444 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 05:01:33.719473 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0806 05:01:33.722729 (+ 3256us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 05:01:33.723713 (+ 984us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 05:01:33.723717 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 05:01:33.723719 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 05:01:33.723847 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 05:01:33.724345 (+ 498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 05:01:33.724348 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 05:01:33.724350 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 05:01:33.724735 (+ 385us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 05:01:33.724891 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0806 05:01:33.725643 (+ 752us) server_negotiation.cc:299] Negotiation successful
0806 05:01:33.725888 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":54}
I20260806 05:01:33.727201 2131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:01:33.707682 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:34193 (local address 127.9.96.67:48865)
0806 05:01:33.708397 (+ 715us) negotiation.cc:107] Waiting for socket to connect
0806 05:01:33.708442 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0806 05:01:33.710112 (+ 1670us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 05:01:33.710842 (+ 730us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 05:01:33.710850 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 05:01:33.711465 (+ 615us) client_negotiation.cc:189] Negotiated authn=SASL
0806 05:01:33.712400 (+ 935us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 05:01:33.712425 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 05:01:33.713868 (+ 1443us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 05:01:33.713874 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 05:01:33.715262 (+ 1388us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 05:01:33.715279 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 05:01:33.716128 (+ 849us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 05:01:33.716139 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 05:01:33.716397 (+ 258us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:01:33.717054 (+ 657us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 05:01:33.717080 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0806 05:01:33.719195 (+ 2115us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 05:01:33.722929 (+ 3734us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 05:01:33.722940 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 05:01:33.722957 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0806 05:01:33.723590 (+ 633us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 05:01:33.723973 (+ 383us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 05:01:33.723977 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 05:01:33.723981 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 05:01:33.724221 (+ 240us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 05:01:33.724873 (+ 652us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 05:01:33.724879 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 05:01:33.725162 (+ 283us) client_negotiation.cc:769] Sending connection context
0806 05:01:33.725497 (+ 335us) client_negotiation.cc:240] Negotiation successful
0806 05:01:33.725961 (+ 464us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":476,"thread_start_us":288,"threads_started":1}
I20260806 05:01:33.727566 2134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:01:33.712115 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35450 (local address 127.9.96.126:34193)
0806 05:01:33.713006 (+ 891us) server_negotiation.cc:206] Beginning negotiation
0806 05:01:33.713012 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 05:01:33.713029 (+ 17us) server_negotiation.cc:407] Connection header received
0806 05:01:33.713084 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 05:01:33.713089 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 05:01:33.713153 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 05:01:33.713265 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0806 05:01:33.714884 (+ 1619us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:01:33.715822 (+ 938us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:01:33.717492 (+ 1670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:01:33.717924 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:01:33.718058 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:01:33.720980 (+ 2922us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 05:01:33.720988 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 05:01:33.720993 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 05:01:33.721052 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0806 05:01:33.724492 (+ 3440us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 05:01:33.725253 (+ 761us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 05:01:33.725258 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 05:01:33.725261 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 05:01:33.725440 (+ 179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 05:01:33.725866 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 05:01:33.725872 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 05:01:33.725876 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 05:01:33.726255 (+ 379us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 05:01:33.726548 (+ 293us) server_negotiation.cc:1035] Waiting for connection context
0806 05:01:33.726890 (+ 342us) server_negotiation.cc:299] Negotiation successful
0806 05:01:33.727107 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":790,"thread_start_us":656,"threads_started":1}
I20260806 05:01:33.728704 2129 heartbeater.cc:347] Connected to a master server at 127.9.96.126:34193
I20260806 05:01:33.729029 2129 heartbeater.cc:464] Registering TS with master...
I20260806 05:01:33.729748 2129 heartbeater.cc:511] Master 127.9.96.126:34193 requested a full tablet report, sending...
I20260806 05:01:33.731172 1485 ts_manager.cc:194] Registered new tserver with Master: 6ebbb98128aa4bd69728676853db94a2 (127.9.96.67:41539)
I20260806 05:01:33.732090 1485 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.96.67@KRBTEST.COM'} at 127.9.96.67:48865
I20260806 05:01:33.742359 9601 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 05:01:33.749408 2134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:01:33.743451 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35464 (local address 127.9.96.126:34193)
0806 05:01:33.743684 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0806 05:01:33.743691 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 05:01:33.744049 (+ 358us) server_negotiation.cc:407] Connection header received
0806 05:01:33.744282 (+ 233us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 05:01:33.744287 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 05:01:33.744452 (+ 165us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 05:01:33.744604 (+ 152us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 05:01:33.745050 (+ 446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:01:33.745760 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:01:33.747196 (+ 1436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:01:33.747792 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:01:33.747928 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:01:33.748425 (+ 497us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 05:01:33.748693 (+ 268us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0806 05:01:33.749105 (+ 412us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:35464: token expired
Metrics: {"server-negotiator.queue_time_us":57}
W20260806 05:01:33.749544 2134 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:35464: token expired
W20260806 05:01:33.750007 1373 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.9.96.126:34193)
I20260806 05:01:33.763238 2134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:01:33.750535 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35472 (local address 127.9.96.126:34193)
0806 05:01:33.750697 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0806 05:01:33.750702 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 05:01:33.750796 (+ 94us) server_negotiation.cc:407] Connection header received
0806 05:01:33.750987 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 05:01:33.750991 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 05:01:33.751057 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 05:01:33.751152 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0806 05:01:33.752658 (+ 1506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:01:33.753352 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:01:33.754965 (+ 1613us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:01:33.755442 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:01:33.755574 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:01:33.758154 (+ 2580us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 05:01:33.758160 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 05:01:33.758165 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 05:01:33.758206 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0806 05:01:33.761037 (+ 2831us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 05:01:33.761618 (+ 581us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 05:01:33.761622 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 05:01:33.761624 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 05:01:33.761755 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 05:01:33.762066 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 05:01:33.762069 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 05:01:33.762071 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 05:01:33.762351 (+ 280us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 05:01:33.762525 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0806 05:01:33.762765 (+ 240us) server_negotiation.cc:299] Negotiation successful
0806 05:01:33.762917 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260806 05:01:33.766311 1485 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35472:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 05:01:33.773852 1485 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 05:01:33.790252 1742 heartbeater.cc:503] Master 127.9.96.126:34193 was elected leader, sending a full tablet report...
I20260806 05:01:33.813081 2149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:01:33.785788 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.67:41539 (local address 127.0.0.1:53556)
0806 05:01:33.802268 (+ 16480us) negotiation.cc:107] Waiting for socket to connect
0806 05:01:33.802306 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0806 05:01:33.802480 (+ 174us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 05:01:33.803881 (+ 1401us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 05:01:33.803892 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 05:01:33.803942 (+ 50us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 05:01:33.804503 (+ 561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 05:01:33.804520 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 05:01:33.807739 (+ 3219us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 05:01:33.807743 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 05:01:33.810049 (+ 2306us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 05:01:33.810066 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 05:01:33.811969 (+ 1903us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 05:01:33.811975 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 05:01:33.812271 (+ 296us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:01:33.812365 (+ 94us) client_negotiation.cc:769] Sending connection context
0806 05:01:33.812504 (+ 139us) client_negotiation.cc:240] Negotiation successful
0806 05:01:33.812693 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":16240,"thread_start_us":20832,"threads_started":1}
I20260806 05:01:33.813091 2144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:01:33.786006 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53556 (local address 127.9.96.67:41539)
0806 05:01:33.801429 (+ 15423us) server_negotiation.cc:206] Beginning negotiation
0806 05:01:33.801442 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 05:01:33.802728 (+ 1286us) server_negotiation.cc:407] Connection header received
0806 05:01:33.802864 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 05:01:33.802875 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 05:01:33.803213 (+ 338us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 05:01:33.803471 (+ 258us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 05:01:33.805409 (+ 1938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:01:33.807058 (+ 1649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:01:33.810208 (+ 3150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:01:33.811715 (+ 1507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:01:33.811875 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:01:33.812122 (+ 247us) server_negotiation.cc:1035] Waiting for connection context
0806 05:01:33.812571 (+ 449us) server_negotiation.cc:299] Negotiation successful
0806 05:01:33.812742 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15179,"thread_start_us":15453,"threads_started":1}
I20260806 05:01:33.816308 2148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:01:33.786225 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54830 (local address 127.9.96.66:36483)
0806 05:01:33.807354 (+ 21129us) server_negotiation.cc:206] Beginning negotiation
0806 05:01:33.807360 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 05:01:33.807437 (+ 77us) server_negotiation.cc:407] Connection header received
0806 05:01:33.807600 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 05:01:33.807608 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 05:01:33.807909 (+ 301us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 05:01:33.808101 (+ 192us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 05:01:33.809017 (+ 916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:01:33.810685 (+ 1668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:01:33.813491 (+ 2806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:01:33.814964 (+ 1473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:01:33.815100 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:01:33.815297 (+ 197us) server_negotiation.cc:1035] Waiting for connection context
0806 05:01:33.815805 (+ 508us) server_negotiation.cc:299] Negotiation successful
0806 05:01:33.815962 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20955,"thread_start_us":20752,"threads_started":1}
I20260806 05:01:33.816898 2146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:01:33.785982 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.66:36483 (local address 127.0.0.1:54830)
0806 05:01:33.805114 (+ 19132us) negotiation.cc:107] Waiting for socket to connect
0806 05:01:33.805131 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 05:01:33.805241 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 05:01:33.808351 (+ 3110us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 05:01:33.808358 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 05:01:33.808387 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 05:01:33.808802 (+ 415us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 05:01:33.808820 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 05:01:33.810843 (+ 2023us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 05:01:33.810848 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 05:01:33.813271 (+ 2423us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 05:01:33.813289 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 05:01:33.815185 (+ 1896us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 05:01:33.815192 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 05:01:33.815377 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:01:33.815436 (+ 59us) client_negotiation.cc:769] Sending connection context
0806 05:01:33.815560 (+ 124us) client_negotiation.cc:240] Negotiation successful
0806 05:01:33.816551 (+ 991us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":19010,"thread_start_us":18880,"threads_started":1}
I20260806 05:01:33.816954 2150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:01:33.789155 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48706 (local address 127.9.96.65:45547)
0806 05:01:33.807882 (+ 18727us) server_negotiation.cc:206] Beginning negotiation
0806 05:01:33.807890 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 05:01:33.807918 (+ 28us) server_negotiation.cc:407] Connection header received
0806 05:01:33.808014 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 05:01:33.808021 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 05:01:33.808235 (+ 214us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 05:01:33.808421 (+ 186us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 05:01:33.809403 (+ 982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:01:33.810956 (+ 1553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:01:33.814109 (+ 3153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:01:33.815623 (+ 1514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:01:33.815747 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:01:33.816012 (+ 265us) server_negotiation.cc:1035] Waiting for connection context
0806 05:01:33.816447 (+ 435us) server_negotiation.cc:299] Negotiation successful
0806 05:01:33.816628 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18539,"thread_start_us":19084,"threads_started":1}
I20260806 05:01:33.817000 2142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:01:33.788374 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.65:45547 (local address 127.0.0.1:48706)
0806 05:01:33.807511 (+ 19137us) negotiation.cc:107] Waiting for socket to connect
0806 05:01:33.807529 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 05:01:33.807642 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 05:01:33.808647 (+ 1005us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 05:01:33.808651 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 05:01:33.808670 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 05:01:33.809185 (+ 515us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 05:01:33.809197 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 05:01:33.811126 (+ 1929us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 05:01:33.811131 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 05:01:33.813946 (+ 2815us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 05:01:33.813960 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 05:01:33.816149 (+ 2189us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 05:01:33.816153 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 05:01:33.816268 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:01:33.816290 (+ 22us) client_negotiation.cc:769] Sending connection context
0806 05:01:33.816401 (+ 111us) client_negotiation.cc:240] Negotiation successful
0806 05:01:33.816623 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":18375,"thread_start_us":13492,"threads_started":1}
I20260806 05:01:33.817591 2037 tablet_service.cc:1467] Processing CreateTablet for tablet b3b5d6e79b3640d99a6644854175b9a9 (DEFAULT_TABLE table=test-table [id=55ef543f3a8a43d1a6b228ce37ff75db]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 05:01:33.819120 2037 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b3b5d6e79b3640d99a6644854175b9a9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 05:01:33.819892 1845 tablet_service.cc:1467] Processing CreateTablet for tablet b3b5d6e79b3640d99a6644854175b9a9 (DEFAULT_TABLE table=test-table [id=55ef543f3a8a43d1a6b228ce37ff75db]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 05:01:33.820878 1845 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b3b5d6e79b3640d99a6644854175b9a9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 05:01:33.820469 1657 tablet_service.cc:1467] Processing CreateTablet for tablet b3b5d6e79b3640d99a6644854175b9a9 (DEFAULT_TABLE table=test-table [id=55ef543f3a8a43d1a6b228ce37ff75db]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 05:01:33.821868 1657 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b3b5d6e79b3640d99a6644854175b9a9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 05:01:33.832046 2151 tablet_bootstrap.cc:492] T b3b5d6e79b3640d99a6644854175b9a9 P 2c79466ec5a14051bbc556f5a3852645: Bootstrap starting.
I20260806 05:01:33.834988 2152 tablet_bootstrap.cc:492] T b3b5d6e79b3640d99a6644854175b9a9 P 6ebbb98128aa4bd69728676853db94a2: Bootstrap starting.
I20260806 05:01:33.836593 2153 tablet_bootstrap.cc:492] T b3b5d6e79b3640d99a6644854175b9a9 P 14be68f793794acaba3556a8e1ef3c13: Bootstrap starting.
I20260806 05:01:33.836629 2151 tablet_bootstrap.cc:654] T b3b5d6e79b3640d99a6644854175b9a9 P 2c79466ec5a14051bbc556f5a3852645: Neither blocks nor log segments found. Creating new log.
I20260806 05:01:33.838292 2151 log.cc:824] T b3b5d6e79b3640d99a6644854175b9a9 P 2c79466ec5a14051bbc556f5a3852645: Log is configured to *not* fsync() on all Append() calls
I20260806 05:01:33.838604 2152 tablet_bootstrap.cc:654] T b3b5d6e79b3640d99a6644854175b9a9 P 6ebbb98128aa4bd69728676853db94a2: Neither blocks nor log segments found. Creating new log.
I20260806 05:01:33.839583 2153 tablet_bootstrap.cc:654] T b3b5d6e79b3640d99a6644854175b9a9 P 14be68f793794acaba3556a8e1ef3c13: Neither blocks nor log segments found. Creating new log.
I20260806 05:01:33.840161 2152 log.cc:824] T b3b5d6e79b3640d99a6644854175b9a9 P 6ebbb98128aa4bd69728676853db94a2: Log is configured to *not* fsync() on all Append() calls
I20260806 05:01:33.840785 2153 log.cc:824] T b3b5d6e79b3640d99a6644854175b9a9 P 14be68f793794acaba3556a8e1ef3c13: Log is configured to *not* fsync() on all Append() calls
I20260806 05:01:33.843758 2152 tablet_bootstrap.cc:492] T b3b5d6e79b3640d99a6644854175b9a9 P 6ebbb98128aa4bd69728676853db94a2: No bootstrap required, opened a new log
I20260806 05:01:33.844069 2152 ts_tablet_manager.cc:1397] T b3b5d6e79b3640d99a6644854175b9a9 P 6ebbb98128aa4bd69728676853db94a2: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.002s
I20260806 05:01:33.844568 2153 tablet_bootstrap.cc:492] T b3b5d6e79b3640d99a6644854175b9a9 P 14be68f793794acaba3556a8e1ef3c13: No bootstrap required, opened a new log
I20260806 05:01:33.844764 2153 ts_tablet_manager.cc:1397] T b3b5d6e79b3640d99a6644854175b9a9 P 14be68f793794acaba3556a8e1ef3c13: Time spent bootstrapping tablet: real 0.008s user 0.001s sys 0.005s
I20260806 05:01:33.845546 2151 tablet_bootstrap.cc:492] T b3b5d6e79b3640d99a6644854175b9a9 P 2c79466ec5a14051bbc556f5a3852645: No bootstrap required, opened a new log
I20260806 05:01:33.845788 2151 ts_tablet_manager.cc:1397] T b3b5d6e79b3640d99a6644854175b9a9 P 2c79466ec5a14051bbc556f5a3852645: Time spent bootstrapping tablet: real 0.014s user 0.003s sys 0.006s
I20260806 05:01:33.852129 2153 raft_consensus.cc:348] T b3b5d6e79b3640d99a6644854175b9a9 P 14be68f793794acaba3556a8e1ef3c13 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ebbb98128aa4bd69728676853db94a2" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 41539 } } peers { permanent_uuid: "2c79466ec5a14051bbc556f5a3852645" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 36483 } } peers { permanent_uuid: "14be68f793794acaba3556a8e1ef3c13" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 45547 } }
I20260806 05:01:33.852512 2153 raft_consensus.cc:374] T b3b5d6e79b3640d99a6644854175b9a9 P 14be68f793794acaba3556a8e1ef3c13 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 05:01:33.852638 2153 raft_consensus.cc:729] T b3b5d6e79b3640d99a6644854175b9a9 P 14be68f793794acaba3556a8e1ef3c13 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 14be68f793794acaba3556a8e1ef3c13, State: Initialized, Role: FOLLOWER
I20260806 05:01:33.853269 2153 consensus_queue.cc:260] T b3b5d6e79b3640d99a6644854175b9a9 P 14be68f793794acaba3556a8e1ef3c13 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ebbb98128aa4bd69728676853db94a2" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 41539 } } peers { permanent_uuid: "2c79466ec5a14051bbc556f5a3852645" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 36483 } } peers { permanent_uuid: "14be68f793794acaba3556a8e1ef3c13" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 45547 } }
I20260806 05:01:33.854700 2152 raft_consensus.cc:348] T b3b5d6e79b3640d99a6644854175b9a9 P 6ebbb98128aa4bd69728676853db94a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ebbb98128aa4bd69728676853db94a2" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 41539 } } peers { permanent_uuid: "2c79466ec5a14051bbc556f5a3852645" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 36483 } } peers { permanent_uuid: "14be68f793794acaba3556a8e1ef3c13" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 45547 } }
I20260806 05:01:33.855031 2153 ts_tablet_manager.cc:1428] T b3b5d6e79b3640d99a6644854175b9a9 P 14be68f793794acaba3556a8e1ef3c13: Time spent starting tablet: real 0.010s user 0.010s sys 0.000s
I20260806 05:01:33.855124 2152 raft_consensus.cc:374] T b3b5d6e79b3640d99a6644854175b9a9 P 6ebbb98128aa4bd69728676853db94a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 05:01:33.855284 2152 raft_consensus.cc:729] T b3b5d6e79b3640d99a6644854175b9a9 P 6ebbb98128aa4bd69728676853db94a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6ebbb98128aa4bd69728676853db94a2, State: Initialized, Role: FOLLOWER
I20260806 05:01:33.855909 2152 consensus_queue.cc:260] T b3b5d6e79b3640d99a6644854175b9a9 P 6ebbb98128aa4bd69728676853db94a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ebbb98128aa4bd69728676853db94a2" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 41539 } } peers { permanent_uuid: "2c79466ec5a14051bbc556f5a3852645" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 36483 } } peers { permanent_uuid: "14be68f793794acaba3556a8e1ef3c13" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 45547 } }
I20260806 05:01:33.856190 2151 raft_consensus.cc:348] T b3b5d6e79b3640d99a6644854175b9a9 P 2c79466ec5a14051bbc556f5a3852645 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ebbb98128aa4bd69728676853db94a2" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 41539 } } peers { permanent_uuid: "2c79466ec5a14051bbc556f5a3852645" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 36483 } } peers { permanent_uuid: "14be68f793794acaba3556a8e1ef3c13" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 45547 } }
I20260806 05:01:33.856688 2151 raft_consensus.cc:374] T b3b5d6e79b3640d99a6644854175b9a9 P 2c79466ec5a14051bbc556f5a3852645 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 05:01:33.856962 2129 heartbeater.cc:503] Master 127.9.96.126:34193 was elected leader, sending a full tablet report...
I20260806 05:01:33.857573 2152 ts_tablet_manager.cc:1428] T b3b5d6e79b3640d99a6644854175b9a9 P 6ebbb98128aa4bd69728676853db94a2: Time spent starting tablet: real 0.013s user 0.011s sys 0.002s
I20260806 05:01:33.858042 2151 raft_consensus.cc:729] T b3b5d6e79b3640d99a6644854175b9a9 P 2c79466ec5a14051bbc556f5a3852645 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c79466ec5a14051bbc556f5a3852645, State: Initialized, Role: FOLLOWER
I20260806 05:01:33.858644 2151 consensus_queue.cc:260] T b3b5d6e79b3640d99a6644854175b9a9 P 2c79466ec5a14051bbc556f5a3852645 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ebbb98128aa4bd69728676853db94a2" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 41539 } } peers { permanent_uuid: "2c79466ec5a14051bbc556f5a3852645" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 36483 } } peers { permanent_uuid: "14be68f793794acaba3556a8e1ef3c13" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 45547 } }
I20260806 05:01:33.859807 1944 heartbeater.cc:503] Master 127.9.96.126:34193 was elected leader, sending a full tablet report...
I20260806 05:01:33.862337 2151 ts_tablet_manager.cc:1428] T b3b5d6e79b3640d99a6644854175b9a9 P 2c79466ec5a14051bbc556f5a3852645: Time spent starting tablet: real 0.016s user 0.009s sys 0.005s
I20260806 05:01:33.895569 2159 raft_consensus.cc:482] T b3b5d6e79b3640d99a6644854175b9a9 P 14be68f793794acaba3556a8e1ef3c13 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 05:01:33.895877 2159 raft_consensus.cc:504] T b3b5d6e79b3640d99a6644854175b9a9 P 14be68f793794acaba3556a8e1ef3c13 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ebbb98128aa4bd69728676853db94a2" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 41539 } } peers { permanent_uuid: "2c79466ec5a14051bbc556f5a3852645" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 36483 } } peers { permanent_uuid: "14be68f793794acaba3556a8e1ef3c13" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 45547 } }
I20260806 05:01:33.897608 2159 leader_election.cc:288] T b3b5d6e79b3640d99a6644854175b9a9 P 14be68f793794acaba3556a8e1ef3c13 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6ebbb98128aa4bd69728676853db94a2 (127.9.96.67:41539), 2c79466ec5a14051bbc556f5a3852645 (127.9.96.66:36483)
I20260806 05:01:33.904537 2148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:01:33.898092 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.96.65:45577 (local address 127.9.96.66:36483)
0806 05:01:33.898307 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0806 05:01:33.898313 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 05:01:33.899016 (+ 703us) server_negotiation.cc:407] Connection header received
0806 05:01:33.899211 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 05:01:33.899215 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 05:01:33.899273 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 05:01:33.899419 (+ 146us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 05:01:33.900114 (+ 695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:01:33.900869 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:01:33.902735 (+ 1866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:01:33.903572 (+ 837us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:01:33.903737 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:01:33.903829 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0806 05:01:33.904124 (+ 295us) server_negotiation.cc:299] Negotiation successful
0806 05:01:33.904246 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260806 05:01:33.904539 2144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:01:33.898039 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.96.65:53413 (local address 127.9.96.67:41539)
0806 05:01:33.898246 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0806 05:01:33.898254 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 05:01:33.898959 (+ 705us) server_negotiation.cc:407] Connection header received
0806 05:01:33.899169 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 05:01:33.899175 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 05:01:33.899243 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 05:01:33.899311 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 05:01:33.900020 (+ 709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:01:33.900788 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:01:33.902693 (+ 1905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:01:33.903547 (+ 854us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:01:33.903638 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:01:33.903773 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0806 05:01:33.904123 (+ 350us) server_negotiation.cc:299] Negotiation successful
0806 05:01:33.904247 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260806 05:01:33.904702 2163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:01:33.897812 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.67:41539 (local address 127.9.96.65:53413)
0806 05:01:33.898821 (+ 1009us) negotiation.cc:107] Waiting for socket to connect
0806 05:01:33.898844 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0806 05:01:33.899012 (+ 168us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 05:01:33.899386 (+ 374us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 05:01:33.899394 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 05:01:33.899422 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 05:01:33.899865 (+ 443us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 05:01:33.899877 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 05:01:33.900938 (+ 1061us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 05:01:33.900946 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 05:01:33.902529 (+ 1583us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 05:01:33.902542 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 05:01:33.903776 (+ 1234us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 05:01:33.903783 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 05:01:33.903958 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:01:33.903984 (+ 26us) client_negotiation.cc:769] Sending connection context
0806 05:01:33.904095 (+ 111us) client_negotiation.cc:240] Negotiation successful
0806 05:01:33.904230 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":811,"mutex_wait_us":27,"thread_start_us":466,"threads_started":1}
I20260806 05:01:33.904829 2164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:01:33.898076 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.66:36483 (local address 127.9.96.65:45577)
0806 05:01:33.898864 (+ 788us) negotiation.cc:107] Waiting for socket to connect
0806 05:01:33.898878 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 05:01:33.899041 (+ 163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 05:01:33.899456 (+ 415us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 05:01:33.899462 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 05:01:33.899500 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 05:01:33.899923 (+ 423us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 05:01:33.899957 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 05:01:33.901012 (+ 1055us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 05:01:33.901016 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 05:01:33.902547 (+ 1531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 05:01:33.902559 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 05:01:33.903840 (+ 1281us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 05:01:33.903845 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 05:01:33.903964 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:01:33.903987 (+ 23us) client_negotiation.cc:769] Sending connection context
0806 05:01:33.904095 (+ 108us) client_negotiation.cc:240] Negotiation successful
0806 05:01:33.904240 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":690,"mutex_wait_us":99,"thread_start_us":513,"threads_started":1}
I20260806 05:01:33.905792 2073 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b3b5d6e79b3640d99a6644854175b9a9" candidate_uuid: "14be68f793794acaba3556a8e1ef3c13" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ebbb98128aa4bd69728676853db94a2" is_pre_election: true
I20260806 05:01:33.905900 1868 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b3b5d6e79b3640d99a6644854175b9a9" candidate_uuid: "14be68f793794acaba3556a8e1ef3c13" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c79466ec5a14051bbc556f5a3852645" is_pre_election: true
I20260806 05:01:33.906363 2073 raft_consensus.cc:2454] T b3b5d6e79b3640d99a6644854175b9a9 P 6ebbb98128aa4bd69728676853db94a2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 14be68f793794acaba3556a8e1ef3c13 in term 0.
I20260806 05:01:33.906454 1868 raft_consensus.cc:2454] T b3b5d6e79b3640d99a6644854175b9a9 P 2c79466ec5a14051bbc556f5a3852645 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 14be68f793794acaba3556a8e1ef3c13 in term 0.
I20260806 05:01:33.907078 1589 leader_election.cc:302] T b3b5d6e79b3640d99a6644854175b9a9 P 14be68f793794acaba3556a8e1ef3c13 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 14be68f793794acaba3556a8e1ef3c13, 6ebbb98128aa4bd69728676853db94a2; no voters:
I20260806 05:01:33.907445 2159 raft_consensus.cc:2781] T b3b5d6e79b3640d99a6644854175b9a9 P 14be68f793794acaba3556a8e1ef3c13 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 05:01:33.907615 2159 raft_consensus.cc:482] T b3b5d6e79b3640d99a6644854175b9a9 P 14be68f793794acaba3556a8e1ef3c13 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 05:01:33.907749 2159 raft_consensus.cc:3037] T b3b5d6e79b3640d99a6644854175b9a9 P 14be68f793794acaba3556a8e1ef3c13 [term 0 FOLLOWER]: Advancing to term 1
I20260806 05:01:33.909955 2159 raft_consensus.cc:504] T b3b5d6e79b3640d99a6644854175b9a9 P 14be68f793794acaba3556a8e1ef3c13 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ebbb98128aa4bd69728676853db94a2" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 41539 } } peers { permanent_uuid: "2c79466ec5a14051bbc556f5a3852645" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 36483 } } peers { permanent_uuid: "14be68f793794acaba3556a8e1ef3c13" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 45547 } }
I20260806 05:01:33.910616 2159 leader_election.cc:288] T b3b5d6e79b3640d99a6644854175b9a9 P 14be68f793794acaba3556a8e1ef3c13 [CANDIDATE]: Term 1 election: Requested vote from peers 6ebbb98128aa4bd69728676853db94a2 (127.9.96.67:41539), 2c79466ec5a14051bbc556f5a3852645 (127.9.96.66:36483)
I20260806 05:01:33.911003 2073 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b3b5d6e79b3640d99a6644854175b9a9" candidate_uuid: "14be68f793794acaba3556a8e1ef3c13" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ebbb98128aa4bd69728676853db94a2"
I20260806 05:01:33.911136 1868 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b3b5d6e79b3640d99a6644854175b9a9" candidate_uuid: "14be68f793794acaba3556a8e1ef3c13" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c79466ec5a14051bbc556f5a3852645"
I20260806 05:01:33.911206 2073 raft_consensus.cc:3037] T b3b5d6e79b3640d99a6644854175b9a9 P 6ebbb98128aa4bd69728676853db94a2 [term 0 FOLLOWER]: Advancing to term 1
I20260806 05:01:33.911399 1868 raft_consensus.cc:3037] T b3b5d6e79b3640d99a6644854175b9a9 P 2c79466ec5a14051bbc556f5a3852645 [term 0 FOLLOWER]: Advancing to term 1
I20260806 05:01:33.914108 2073 raft_consensus.cc:2454] T b3b5d6e79b3640d99a6644854175b9a9 P 6ebbb98128aa4bd69728676853db94a2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 14be68f793794acaba3556a8e1ef3c13 in term 1.
I20260806 05:01:33.914119 1868 raft_consensus.cc:2454] T b3b5d6e79b3640d99a6644854175b9a9 P 2c79466ec5a14051bbc556f5a3852645 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 14be68f793794acaba3556a8e1ef3c13 in term 1.
I20260806 05:01:33.914506 1589 leader_election.cc:302] T b3b5d6e79b3640d99a6644854175b9a9 P 14be68f793794acaba3556a8e1ef3c13 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 14be68f793794acaba3556a8e1ef3c13, 6ebbb98128aa4bd69728676853db94a2; no voters:
I20260806 05:01:33.914810 2159 raft_consensus.cc:2781] T b3b5d6e79b3640d99a6644854175b9a9 P 14be68f793794acaba3556a8e1ef3c13 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 05:01:33.915339 2159 raft_consensus.cc:686] T b3b5d6e79b3640d99a6644854175b9a9 P 14be68f793794acaba3556a8e1ef3c13 [term 1 LEADER]: Becoming Leader. State: Replica: 14be68f793794acaba3556a8e1ef3c13, State: Running, Role: LEADER
I20260806 05:01:33.915870 2159 consensus_queue.cc:237] T b3b5d6e79b3640d99a6644854175b9a9 P 14be68f793794acaba3556a8e1ef3c13 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ebbb98128aa4bd69728676853db94a2" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 41539 } } peers { permanent_uuid: "2c79466ec5a14051bbc556f5a3852645" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 36483 } } peers { permanent_uuid: "14be68f793794acaba3556a8e1ef3c13" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 45547 } }
I20260806 05:01:33.920578 1485 catalog_manager.cc:5495] T b3b5d6e79b3640d99a6644854175b9a9 P 14be68f793794acaba3556a8e1ef3c13 reported cstate change: term changed from 0 to 1, leader changed from <none> to 14be68f793794acaba3556a8e1ef3c13 (127.9.96.65). New cstate: current_term: 1 leader_uuid: "14be68f793794acaba3556a8e1ef3c13" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ebbb98128aa4bd69728676853db94a2" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 41539 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c79466ec5a14051bbc556f5a3852645" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 36483 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "14be68f793794acaba3556a8e1ef3c13" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 45547 } health_report { overall_health: HEALTHY } } }
I20260806 05:01:33.950441 2150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:01:33.944725 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48712 (local address 127.9.96.65:45547)
0806 05:01:33.944956 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0806 05:01:33.944965 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 05:01:33.944991 (+ 26us) server_negotiation.cc:407] Connection header received
0806 05:01:33.945079 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 05:01:33.945083 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 05:01:33.945167 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 05:01:33.945345 (+ 178us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 05:01:33.945717 (+ 372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:01:33.946457 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:01:33.948137 (+ 1680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:01:33.948566 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:01:33.948714 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:01:33.949330 (+ 616us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 05:01:33.949831 (+ 501us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 05:01:33.949998 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0806 05:01:33.950082 (+ 84us) server_negotiation.cc:299] Negotiation successful
0806 05:01:33.950167 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
W20260806 05:01:33.956180 2130 tablet.cc:2369] T b3b5d6e79b3640d99a6644854175b9a9 P 6ebbb98128aa4bd69728676853db94a2: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 05:01:33.958724 2073 raft_consensus.cc:1261] T b3b5d6e79b3640d99a6644854175b9a9 P 6ebbb98128aa4bd69728676853db94a2 [term 1 FOLLOWER]: Refusing update from remote peer 14be68f793794acaba3556a8e1ef3c13: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 05:01:33.958757 1868 raft_consensus.cc:1261] T b3b5d6e79b3640d99a6644854175b9a9 P 2c79466ec5a14051bbc556f5a3852645 [term 1 FOLLOWER]: Refusing update from remote peer 14be68f793794acaba3556a8e1ef3c13: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 05:01:33.959904 2159 consensus_queue.cc:1035] T b3b5d6e79b3640d99a6644854175b9a9 P 14be68f793794acaba3556a8e1ef3c13 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ebbb98128aa4bd69728676853db94a2" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 41539 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 05:01:33.960232 2165 consensus_queue.cc:1035] T b3b5d6e79b3640d99a6644854175b9a9 P 14be68f793794acaba3556a8e1ef3c13 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c79466ec5a14051bbc556f5a3852645" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 36483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 05:01:33.987591 1486 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35472:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 05:01:33.987859 1486 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35472:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 05:01:33.991263 1486 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 28af3f4cbb8141709b39138ff8f369b3: Sending DeleteTablet for 3 replicas of tablet b3b5d6e79b3640d99a6644854175b9a9
I20260806 05:01:33.992461 2037 tablet_service.cc:1514] Processing DeleteTablet for tablet b3b5d6e79b3640d99a6644854175b9a9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 05:01:33 UTC) from {username='kudu', principal='kudu/127.9.96.126@KRBTEST.COM'} at 127.0.0.1:53556
I20260806 05:01:33.992482 1845 tablet_service.cc:1514] Processing DeleteTablet for tablet b3b5d6e79b3640d99a6644854175b9a9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 05:01:33 UTC) from {username='kudu', principal='kudu/127.9.96.126@KRBTEST.COM'} at 127.0.0.1:54830
I20260806 05:01:33.992489 1657 tablet_service.cc:1514] Processing DeleteTablet for tablet b3b5d6e79b3640d99a6644854175b9a9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 05:01:33 UTC) from {username='kudu', principal='kudu/127.9.96.126@KRBTEST.COM'} at 127.0.0.1:48706
I20260806 05:01:33.994369 9601 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu pid 1538
I20260806 05:01:34.005370 2186 tablet_replica.cc:330] T b3b5d6e79b3640d99a6644854175b9a9 P 6ebbb98128aa4bd69728676853db94a2: stopping tablet replica
I20260806 05:01:34.006770 2184 tablet_replica.cc:330] T b3b5d6e79b3640d99a6644854175b9a9 P 14be68f793794acaba3556a8e1ef3c13: stopping tablet replica
I20260806 05:01:34.006748 2186 raft_consensus.cc:2229] T b3b5d6e79b3640d99a6644854175b9a9 P 6ebbb98128aa4bd69728676853db94a2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 05:01:34.007360 2186 raft_consensus.cc:2258] T b3b5d6e79b3640d99a6644854175b9a9 P 6ebbb98128aa4bd69728676853db94a2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 05:01:34.007792 2184 raft_consensus.cc:2229] T b3b5d6e79b3640d99a6644854175b9a9 P 14be68f793794acaba3556a8e1ef3c13 [term 1 LEADER]: Raft consensus shutting down.
I20260806 05:01:34.008592 2184 raft_consensus.cc:2258] T b3b5d6e79b3640d99a6644854175b9a9 P 14be68f793794acaba3556a8e1ef3c13 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 05:01:34.009863 2186 ts_tablet_manager.cc:1905] T b3b5d6e79b3640d99a6644854175b9a9 P 6ebbb98128aa4bd69728676853db94a2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 05:01:34.010778 2184 ts_tablet_manager.cc:1905] T b3b5d6e79b3640d99a6644854175b9a9 P 14be68f793794acaba3556a8e1ef3c13: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 05:01:34.011179 2183 tablet_replica.cc:330] T b3b5d6e79b3640d99a6644854175b9a9 P 2c79466ec5a14051bbc556f5a3852645: stopping tablet replica
I20260806 05:01:34.011953 2183 raft_consensus.cc:2229] T b3b5d6e79b3640d99a6644854175b9a9 P 2c79466ec5a14051bbc556f5a3852645 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 05:01:34.013084 2183 raft_consensus.cc:2258] T b3b5d6e79b3640d99a6644854175b9a9 P 2c79466ec5a14051bbc556f5a3852645 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 05:01:34.017379 2183 ts_tablet_manager.cc:1905] T b3b5d6e79b3640d99a6644854175b9a9 P 2c79466ec5a14051bbc556f5a3852645: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 05:01:34.017529 2186 ts_tablet_manager.cc:1918] T b3b5d6e79b3640d99a6644854175b9a9 P 6ebbb98128aa4bd69728676853db94a2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 05:01:34.017625 2150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:01:33.995306 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48722 (local address 127.9.96.65:45547)
0806 05:01:33.995991 (+ 685us) server_negotiation.cc:206] Beginning negotiation
0806 05:01:33.996001 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 05:01:33.996022 (+ 21us) server_negotiation.cc:407] Connection header received
0806 05:01:33.996089 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 05:01:33.996097 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 05:01:33.996208 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 05:01:33.996325 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0806 05:01:33.998865 (+ 2540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:01:34.000491 (+ 1626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:01:34.003074 (+ 2583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:01:34.004093 (+ 1019us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:01:34.004259 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:01:34.008644 (+ 4385us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 05:01:34.008661 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 05:01:34.008679 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 05:01:34.008725 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0806 05:01:34.013111 (+ 4386us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 05:01:34.014090 (+ 979us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 05:01:34.014099 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 05:01:34.014103 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 05:01:34.014308 (+ 205us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 05:01:34.014808 (+ 500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 05:01:34.014814 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 05:01:34.014817 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 05:01:34.015424 (+ 607us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 05:01:34.015718 (+ 294us) server_negotiation.cc:1035] Waiting for connection context
0806 05:01:34.016982 (+ 1264us) server_negotiation.cc:299] Negotiation successful
0806 05:01:34.017273 (+ 291us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":526}
I20260806 05:01:34.017740 2186 log.cc:1196] T b3b5d6e79b3640d99a6644854175b9a9 P 6ebbb98128aa4bd69728676853db94a2: Deleting WAL directory at /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785992227050689-9601-0/minicluster-data/ts-2/wal/wals/b3b5d6e79b3640d99a6644854175b9a9
I20260806 05:01:34.018587 1730 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 05:01:34.019114 2186 ts_tablet_manager.cc:1939] T b3b5d6e79b3640d99a6644854175b9a9 P 6ebbb98128aa4bd69728676853db94a2: Deleting consensus metadata
I20260806 05:01:34.021090 1459 catalog_manager.cc:4841] TS 6ebbb98128aa4bd69728676853db94a2 (127.9.96.67:41539): tablet b3b5d6e79b3640d99a6644854175b9a9 (table test-table [id=55ef543f3a8a43d1a6b228ce37ff75db]) successfully deleted
I20260806 05:01:34.025731 2183 ts_tablet_manager.cc:1918] T b3b5d6e79b3640d99a6644854175b9a9 P 2c79466ec5a14051bbc556f5a3852645: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 05:01:34.025925 2183 log.cc:1196] T b3b5d6e79b3640d99a6644854175b9a9 P 2c79466ec5a14051bbc556f5a3852645: Deleting WAL directory at /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785992227050689-9601-0/minicluster-data/ts-1/wal/wals/b3b5d6e79b3640d99a6644854175b9a9
I20260806 05:01:34.026413 2183 ts_tablet_manager.cc:1939] T b3b5d6e79b3640d99a6644854175b9a9 P 2c79466ec5a14051bbc556f5a3852645: Deleting consensus metadata
I20260806 05:01:34.027932 1462 catalog_manager.cc:4841] TS 2c79466ec5a14051bbc556f5a3852645 (127.9.96.66:36483): tablet b3b5d6e79b3640d99a6644854175b9a9 (table test-table [id=55ef543f3a8a43d1a6b228ce37ff75db]) successfully deleted
I20260806 05:01:34.116619 9601 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu with pid 1538
I20260806 05:01:34.141196 9601 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu pid 1746
W20260806 05:01:34.141285 1461 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.9.96.65:45547 (error 108)
I20260806 05:01:34.143465 2142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:01:34.142931 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.65:45547 (local address 127.0.0.1:48734)
0806 05:01:34.143126 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0806 05:01:34.143207 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.96.65:45547: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260806 05:01:34.143668 1461 catalog_manager.cc:4584] TS 14be68f793794acaba3556a8e1ef3c13 (127.9.96.65:45547): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet b3b5d6e79b3640d99a6644854175b9a9: Network error: Client connection negotiation failed: client connection to 127.9.96.65:45547: connect: Connection refused (error 111)
I20260806 05:01:34.155680 2148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:01:34.142054 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54844 (local address 127.9.96.66:36483)
0806 05:01:34.142266 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0806 05:01:34.142272 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 05:01:34.142355 (+ 83us) server_negotiation.cc:407] Connection header received
0806 05:01:34.142591 (+ 236us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 05:01:34.142596 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 05:01:34.142664 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 05:01:34.142773 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0806 05:01:34.144477 (+ 1704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:01:34.145210 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:01:34.146587 (+ 1377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:01:34.146986 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:01:34.147131 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:01:34.149967 (+ 2836us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 05:01:34.149981 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 05:01:34.149996 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 05:01:34.150039 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0806 05:01:34.152747 (+ 2708us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 05:01:34.153472 (+ 725us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 05:01:34.153480 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 05:01:34.153482 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 05:01:34.153614 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 05:01:34.153993 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 05:01:34.153998 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 05:01:34.154002 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 05:01:34.154412 (+ 410us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 05:01:34.154748 (+ 336us) server_negotiation.cc:1035] Waiting for connection context
0806 05:01:34.155078 (+ 330us) server_negotiation.cc:299] Negotiation successful
0806 05:01:34.155320 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260806 05:01:34.156371 1939 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 05:01:34.197391 2142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:01:34.196834 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.65:45547 (local address 127.0.0.1:48744)
0806 05:01:34.197095 (+ 261us) negotiation.cc:107] Waiting for socket to connect
0806 05:01:34.197193 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.96.65:45547: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118}
I20260806 05:01:34.198428 2142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:01:34.198097 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.65:45547 (local address 127.0.0.1:48756)
0806 05:01:34.198245 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0806 05:01:34.198315 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.96.65:45547: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260806 05:01:34.248137 9601 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu with pid 1746
I20260806 05:01:34.259155 2142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:01:34.258701 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.65:45547 (local address 127.0.0.1:48764)
0806 05:01:34.258881 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0806 05:01:34.258980 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.96.65:45547: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260806 05:01:34.260260 2142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:01:34.259925 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.65:45547 (local address 127.0.0.1:48780)
0806 05:01:34.260043 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0806 05:01:34.260138 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.96.65:45547: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
I20260806 05:01:34.272120 9601 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu pid 1948
I20260806 05:01:34.285662 2144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:01:34.272994 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53562 (local address 127.9.96.67:41539)
0806 05:01:34.273174 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0806 05:01:34.273178 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 05:01:34.273198 (+ 20us) server_negotiation.cc:407] Connection header received
0806 05:01:34.273345 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 05:01:34.273349 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 05:01:34.273427 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 05:01:34.273514 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0806 05:01:34.274856 (+ 1342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:01:34.275695 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:01:34.276972 (+ 1277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:01:34.277366 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:01:34.277460 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:01:34.280138 (+ 2678us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 05:01:34.280172 (+ 34us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 05:01:34.280190 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 05:01:34.280240 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0806 05:01:34.283146 (+ 2906us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 05:01:34.283838 (+ 692us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 05:01:34.283846 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 05:01:34.283849 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 05:01:34.283991 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 05:01:34.284345 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 05:01:34.284349 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 05:01:34.284352 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 05:01:34.284741 (+ 389us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 05:01:34.284984 (+ 243us) server_negotiation.cc:1035] Waiting for connection context
0806 05:01:34.285239 (+ 255us) server_negotiation.cc:299] Negotiation successful
0806 05:01:34.285410 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260806 05:01:34.286319 2123 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 05:01:34.340989 2142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:01:34.340447 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.65:45547 (local address 127.0.0.1:48782)
0806 05:01:34.340701 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0806 05:01:34.340798 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.96.65:45547: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96}
I20260806 05:01:34.342289 2142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:01:34.341817 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.65:45547 (local address 127.0.0.1:48786)
0806 05:01:34.342072 (+ 255us) negotiation.cc:107] Waiting for socket to connect
0806 05:01:34.342180 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.96.65:45547: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":137}
I20260806 05:01:34.376374 9601 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu with pid 1948
I20260806 05:01:34.401713 9601 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu pid 1435
I20260806 05:01:34.402879 1529 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 05:01:34.516178 2142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:01:34.515649 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.65:45547 (local address 127.0.0.1:48790)
0806 05:01:34.515902 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0806 05:01:34.516007 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.96.65:45547: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":89}
I20260806 05:01:34.516834 9601 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu with pid 1435
I20260806 05:01:54.543803 9601 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
/tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785992227050689-9601-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785992227050689-9601-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785992227050689-9601-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785992227050689-9601-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.96.126:34193
--webserver_interface=127.9.96.126
--webserver_port=33641
--keytab_file=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.126.keytab
--principal=kudu/127.9.96.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.96.84:41431
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.96.126:34193
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785992227050689-9601-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785992227050689-9601-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785992227050689-9601-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 05:01:54.883502 2197 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 05:01:54.883793 2197 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 05:01:54.883886 2197 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 05:01:54.891032 2197 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260806 05:01:54.891134 2197 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260806 05:01:54.891175 2197 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260806 05:01:54.891234 2197 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 05:01:54.891290 2197 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 05:01:54.891433 2197 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 05:01:54.891526 2197 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 05:01:54.891637 2197 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 05:01:54.901532 2197 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.96.84:41431
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785992227050689-9601-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785992227050689-9601-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.9.96.126:34193
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.126.keytab
--principal=kudu/127.9.96.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.96.126:34193
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785992227050689-9601-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.96.126
--webserver_port=33641
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785992227050689-9601-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 03:56:12 UTC on ccycloud.quasar-guijbm.root.comops.site
build id 4569618
ASAN enabled
I20260806 05:01:54.902886 2197 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 05:01:54.904273 2197 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 05:01:54.916733 2208 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 05:01:54.916771 2211 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 05:01:54.916795 2209 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 05:01:54.917665 2197 server_base.cc:1034] running on GCE node
I20260806 05:01:54.918454 2197 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 05:01:54.919926 2197 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 05:01:54.921185 2197 hybrid_clock.cc:648] HybridClock initialized: now 1785992514921108 us; error 32 us; skew 500 ppm
I20260806 05:01:54.921597 2197 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 05:01:54 dist-test-slave-x4qh krb5kdc[593](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992514, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.96.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 05:01:54.929032 2197 init.cc:377] Logged in from keytab as kudu/127.9.96.126@KRBTEST.COM (short username kudu)
I20260806 05:01:54.930619 2197 webserver.cc:460] Webserver started at http://127.9.96.126:33641/ using document root <none> and password file <none>
I20260806 05:01:54.931344 2197 fs_manager.cc:359] Metadata directory not provided
I20260806 05:01:54.931547 2197 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 05:01:54.937058 2197 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.001s
I20260806 05:01:54.940016 2224 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 05:01:54.941598 2197 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260806 05:01:54.941803 2197 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785992227050689-9601-0/minicluster-data/master-0/data,/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785992227050689-9601-0/minicluster-data/master-0/wal
uuid: "28af3f4cbb8141709b39138ff8f369b3"
format_stamp: "Formatted at 2026-08-06 05:01:08 on dist-test-slave-x4qh"
server_key: "c36b42d5d25059f8032ec59f36e1c6cf"
server_key_iv: "97e75fb038a44ae429b93be19508ec27"
server_key_version: "encryptionkey@0"
I20260806 05:01:54.942792 2197 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785992227050689-9601-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785992227050689-9601-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785992227050689-9601-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 05:01:54.954108 2197 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 05:01:54.958410 2197 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 05:01:54.958734 2197 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 05:01:54.977955 2197 rpc_server.cc:307] RPC server started. Bound to: 127.9.96.126:34193
I20260806 05:01:54.978011 2295 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.96.126:34193 every 8 connection(s)
I20260806 05:01:54.979523 2197 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785992227050689-9601-0/minicluster-data/master-0/data/info.pb
I20260806 05:01:54.983635 9601 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu as pid 2197
I20260806 05:01:54.984803 9601 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
/tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785992227050689-9601-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785992227050689-9601-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785992227050689-9601-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785992227050689-9601-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.96.65:45547
--local_ip_for_outbound_sockets=127.9.96.65
--tserver_master_addrs=127.9.96.126:34193
--webserver_port=44495
--webserver_interface=127.9.96.65
--keytab_file=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.65.keytab
--principal=kudu/127.9.96.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.96.84:41431
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785992227050689-9601-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785992227050689-9601-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785992227050689-9601-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 05:01:54.989075 2296 sys_catalog.cc:263] Verifying existing consensus state
I20260806 05:01:54.994902 2296 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 28af3f4cbb8141709b39138ff8f369b3: Bootstrap starting.
I20260806 05:01:55.018802 2296 log.cc:824] T 00000000000000000000000000000000 P 28af3f4cbb8141709b39138ff8f369b3: Log is configured to *not* fsync() on all Append() calls
I20260806 05:01:55.036695 2296 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 28af3f4cbb8141709b39138ff8f369b3: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 05:01:55.037312 2296 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 28af3f4cbb8141709b39138ff8f369b3: Bootstrap complete.
I20260806 05:01:55.048358 2296 raft_consensus.cc:348] T 00000000000000000000000000000000 P 28af3f4cbb8141709b39138ff8f369b3 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "28af3f4cbb8141709b39138ff8f369b3" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 34193 } }
I20260806 05:01:55.049556 2296 raft_consensus.cc:729] T 00000000000000000000000000000000 P 28af3f4cbb8141709b39138ff8f369b3 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 28af3f4cbb8141709b39138ff8f369b3, State: Initialized, Role: FOLLOWER
I20260806 05:01:55.050282 2296 consensus_queue.cc:260] T 00000000000000000000000000000000 P 28af3f4cbb8141709b39138ff8f369b3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "28af3f4cbb8141709b39138ff8f369b3" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 34193 } }
I20260806 05:01:55.050534 2296 raft_consensus.cc:388] T 00000000000000000000000000000000 P 28af3f4cbb8141709b39138ff8f369b3 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 05:01:55.050719 2296 raft_consensus.cc:482] T 00000000000000000000000000000000 P 28af3f4cbb8141709b39138ff8f369b3 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 05:01:55.050930 2296 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 28af3f4cbb8141709b39138ff8f369b3 [term 2 FOLLOWER]: Advancing to term 3
I20260806 05:01:55.055109 2296 raft_consensus.cc:504] T 00000000000000000000000000000000 P 28af3f4cbb8141709b39138ff8f369b3 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "28af3f4cbb8141709b39138ff8f369b3" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 34193 } }
I20260806 05:01:55.055744 2296 leader_election.cc:302] T 00000000000000000000000000000000 P 28af3f4cbb8141709b39138ff8f369b3 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 28af3f4cbb8141709b39138ff8f369b3; no voters:
I20260806 05:01:55.056515 2296 leader_election.cc:288] T 00000000000000000000000000000000 P 28af3f4cbb8141709b39138ff8f369b3 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260806 05:01:55.056675 2302 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 28af3f4cbb8141709b39138ff8f369b3 [term 3 FOLLOWER]: Leader election won for term 3
I20260806 05:01:55.057803 2302 raft_consensus.cc:686] T 00000000000000000000000000000000 P 28af3f4cbb8141709b39138ff8f369b3 [term 3 LEADER]: Becoming Leader. State: Replica: 28af3f4cbb8141709b39138ff8f369b3, State: Running, Role: LEADER
I20260806 05:01:55.058399 2302 consensus_queue.cc:237] T 00000000000000000000000000000000 P 28af3f4cbb8141709b39138ff8f369b3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "28af3f4cbb8141709b39138ff8f369b3" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 34193 } }
I20260806 05:01:55.058982 2296 sys_catalog.cc:564] T 00000000000000000000000000000000 P 28af3f4cbb8141709b39138ff8f369b3 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 05:01:55.061296 2304 sys_catalog.cc:455] T 00000000000000000000000000000000 P 28af3f4cbb8141709b39138ff8f369b3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "28af3f4cbb8141709b39138ff8f369b3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "28af3f4cbb8141709b39138ff8f369b3" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 34193 } } }
I20260806 05:01:55.061533 2306 sys_catalog.cc:455] T 00000000000000000000000000000000 P 28af3f4cbb8141709b39138ff8f369b3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 28af3f4cbb8141709b39138ff8f369b3. Latest consensus state: current_term: 3 leader_uuid: "28af3f4cbb8141709b39138ff8f369b3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "28af3f4cbb8141709b39138ff8f369b3" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 34193 } } }
I20260806 05:01:55.061802 2304 sys_catalog.cc:458] T 00000000000000000000000000000000 P 28af3f4cbb8141709b39138ff8f369b3 [sys.catalog]: This master's current role is: LEADER
I20260806 05:01:55.061882 2306 sys_catalog.cc:458] T 00000000000000000000000000000000 P 28af3f4cbb8141709b39138ff8f369b3 [sys.catalog]: This master's current role is: LEADER
I20260806 05:01:55.065482 2310 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 05:01:55.076756 2310 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 05:01:55.078907 2310 catalog_manager.cc:1259] Loaded cluster ID: d13035ae4ef14dfab1dc54b0a8fbf05e
I20260806 05:01:55.079044 2310 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 05:01:55.082484 2310 catalog_manager.cc:1504] Loading token signing keys...
I20260806 05:01:55.085142 2310 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 28af3f4cbb8141709b39138ff8f369b3: Loaded TSK: 0
I20260806 05:01:55.085362 2310 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 28af3f4cbb8141709b39138ff8f369b3: Loaded TSK: 1
F20260806 05:01:55.086884 2310 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f3df059d9ad google::LogMessage::Fail() at ??:0
@ 0x7f3df05a088a google::LogMessage::SendToLog() at ??:0
@ 0x7f3df059d4de google::LogMessage::Flush() at ??:0
@ 0x7f3df059ea19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f3df43017d2 kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7f3df42f073e kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7f3e05fcc1c0 kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7f3e05fcb783 kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7f3e060572a7 kudu::master::CatalogManager::PrepareForLeadershipTask()::$_10::operator()() at ??:0
@ 0x7f3e060570c4 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x7f3e060a15e9 std::function<>::operator()() at ??:0
@ 0x7f3e05fcfcdd kudu::master::CatalogManager::PrepareForLeadershipTask()::$_6::operator()() at ??:0
@ 0x7f3e05fcdc36 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f3e06055610 kudu::master::CatalogManager::ElectedAsLeaderCb()::$_2::operator()() at ??:0
@ 0x7f3e06055439 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f3df31e9345 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f3df31ec91c kudu::ThreadPool::CreateThread()::$_2::operator()() at ??:0
@ 0x7f3df31ec799 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f3df31c7eaf kudu::Thread::SuperviseThread() at ??:0
@ 0x7f3df64a16db start_thread at ??:0
@ 0x7f3def43761f clone at ??:0
W20260806 05:01:55.403015 2298 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 05:01:55.403295 2298 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 05:01:55.403553 2298 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 05:01:55.411432 2298 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 05:01:55.411581 2298 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 05:01:55.411778 2298 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.96.65
I20260806 05:01:55.421578 2298 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.96.84:41431
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785992227050689-9601-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785992227050689-9601-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.65.keytab
--principal=kudu/127.9.96.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.96.65:45547
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785992227050689-9601-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.96.65
--webserver_port=44495
--tserver_master_addrs=127.9.96.126:34193
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.96.65
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785992227050689-9601-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 03:56:12 UTC on ccycloud.quasar-guijbm.root.comops.site
build id 4569618
ASAN enabled
I20260806 05:01:55.422856 2298 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 05:01:55.424227 2298 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 05:01:55.437440 2365 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 05:01:55.437547 2368 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 05:01:55.437469 2366 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 05:01:55.438705 2298 server_base.cc:1034] running on GCE node
I20260806 05:01:55.439471 2298 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 05:01:55.440498 2298 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 05:01:55.441720 2298 hybrid_clock.cc:648] HybridClock initialized: now 1785992515441666 us; error 43 us; skew 500 ppm
I20260806 05:01:55.442063 2298 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 05:01:55 dist-test-slave-x4qh krb5kdc[593](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992515, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.96.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 05:01:55.445287 2298 init.cc:377] Logged in from keytab as kudu/127.9.96.65@KRBTEST.COM (short username kudu)
I20260806 05:01:55.446774 2298 webserver.cc:460] Webserver started at http://127.9.96.65:44495/ using document root <none> and password file <none>
I20260806 05:01:55.447515 2298 fs_manager.cc:359] Metadata directory not provided
I20260806 05:01:55.447650 2298 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 05:01:55.452836 2298 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.002s
I20260806 05:01:55.455921 2375 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 05:01:55.457255 2298 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260806 05:01:55.457458 2298 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785992227050689-9601-0/minicluster-data/ts-0/data,/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785992227050689-9601-0/minicluster-data/ts-0/wal
uuid: "14be68f793794acaba3556a8e1ef3c13"
format_stamp: "Formatted at 2026-08-06 05:01:09 on dist-test-slave-x4qh"
server_key: "3128738e1e7ef5228a1daa36f098d2f4"
server_key_iv: "7922f313fa41aa73b12b8f5bc3e10018"
server_key_version: "encryptionkey@0"
I20260806 05:01:55.458454 2298 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785992227050689-9601-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785992227050689-9601-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785992227050689-9601-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 05:01:55.477234 2298 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 05:01:55.481550 2298 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 05:01:55.481853 2298 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 05:01:55.482920 2298 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 05:01:55.484824 2388 ts_tablet_manager.cc:542] Loading tablet metadata (0/1 complete)
W20260806 05:01:55.490324 2388 ts_tablet_manager.cc:1852] T b3b5d6e79b3640d99a6644854175b9a9 P 14be68f793794acaba3556a8e1ef3c13: Tablet Manager startup: Rolling forward tablet deletion of type TABLET_DATA_DELETED
I20260806 05:01:55.491339 2388 ts_tablet_manager.cc:1905] T b3b5d6e79b3640d99a6644854175b9a9 P 14be68f793794acaba3556a8e1ef3c13: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 05:01:55.498418 2388 ts_tablet_manager.cc:1918] T b3b5d6e79b3640d99a6644854175b9a9 P 14be68f793794acaba3556a8e1ef3c13: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 05:01:55.498646 2388 log.cc:1196] T b3b5d6e79b3640d99a6644854175b9a9 P 14be68f793794acaba3556a8e1ef3c13: Deleting WAL directory at /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785992227050689-9601-0/minicluster-data/ts-0/wal/wals/b3b5d6e79b3640d99a6644854175b9a9
I20260806 05:01:55.499142 2388 ts_tablet_manager.cc:1939] T b3b5d6e79b3640d99a6644854175b9a9 P 14be68f793794acaba3556a8e1ef3c13: Deleting consensus metadata
I20260806 05:01:55.499559 2298 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 0 live tablets)
I20260806 05:01:55.499686 2298 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.015s user 0.001s sys 0.000s
I20260806 05:01:55.499867 2298 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 05:01:55.499941 2298 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 05:01:55.547922 2298 rpc_server.cc:307] RPC server started. Bound to: 127.9.96.65:45547
I20260806 05:01:55.548026 2555 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.96.65:45547 every 8 connection(s)
I20260806 05:01:55.549311 2298 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785992227050689-9601-0/minicluster-data/ts-0/data/info.pb
I20260806 05:01:55.554113 2562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:01:55.552342 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:34193 (local address 127.9.96.65:55107)
0806 05:01:55.553056 (+ 714us) negotiation.cc:107] Waiting for socket to connect
0806 05:01:55.553295 (+ 239us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.96.126:34193: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":476,"thread_start_us":282,"threads_started":1}
W20260806 05:01:55.554774 2558 heartbeater.cc:644] Failed to heartbeat to 127.9.96.126:34193 (0 consecutive failures): Network error: Failed to ping master at 127.9.96.126:34193: Client connection negotiation failed: client connection to 127.9.96.126:34193: connect: Connection refused (error 111)
I20260806 05:01:55.555589 2562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:01:55.555232 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:34193 (local address 127.9.96.65:51245)
0806 05:01:55.555404 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0806 05:01:55.555491 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.96.126:34193: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260806 05:01:55.556460 2562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:01:55.556121 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:34193 (local address 127.9.96.65:42707)
0806 05:01:55.556274 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0806 05:01:55.556350 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.96.126:34193: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260806 05:01:55.556694 2558 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260806 05:01:55.558724 9601 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu as pid 2298
I20260806 05:01:55.559439 9601 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
/tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785992227050689-9601-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785992227050689-9601-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785992227050689-9601-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785992227050689-9601-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.96.66:36483
--local_ip_for_outbound_sockets=127.9.96.66
--tserver_master_addrs=127.9.96.126:34193
--webserver_port=37521
--webserver_interface=127.9.96.66
--keytab_file=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.66.keytab
--principal=kudu/127.9.96.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.96.84:41431
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785992227050689-9601-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785992227050689-9601-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785992227050689-9601-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 05:01:55.898819 2564 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 05:01:55.899091 2564 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 05:01:55.899188 2564 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 05:01:55.906543 2564 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 05:01:55.906652 2564 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 05:01:55.906769 2564 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.96.66
I20260806 05:01:55.916839 2564 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.96.84:41431
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785992227050689-9601-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785992227050689-9601-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.66.keytab
--principal=kudu/127.9.96.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.96.66:36483
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785992227050689-9601-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.96.66
--webserver_port=37521
--tserver_master_addrs=127.9.96.126:34193
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.96.66
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785992227050689-9601-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 03:56:12 UTC on ccycloud.quasar-guijbm.root.comops.site
build id 4569618
ASAN enabled
I20260806 05:01:55.918133 2564 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 05:01:55.919421 2564 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 05:01:55.932878 2571 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 05:01:55.932857 2573 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 05:01:55.932947 2575 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 05:01:55.934590 2564 server_base.cc:1034] running on GCE node
I20260806 05:01:55.935220 2564 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 05:01:55.936323 2564 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 05:01:55.937579 2564 hybrid_clock.cc:648] HybridClock initialized: now 1785992515937541 us; error 78 us; skew 500 ppm
I20260806 05:01:55.937929 2564 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 05:01:55 dist-test-slave-x4qh krb5kdc[593](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992515, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.96.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 05:01:55.941385 2564 init.cc:377] Logged in from keytab as kudu/127.9.96.66@KRBTEST.COM (short username kudu)
I20260806 05:01:55.943011 2564 webserver.cc:460] Webserver started at http://127.9.96.66:37521/ using document root <none> and password file <none>
I20260806 05:01:55.943822 2564 fs_manager.cc:359] Metadata directory not provided
I20260806 05:01:55.943992 2564 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 05:01:55.949671 2564 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.002s sys 0.003s
I20260806 05:01:55.952651 2584 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 05:01:55.954109 2564 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260806 05:01:55.954304 2564 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785992227050689-9601-0/minicluster-data/ts-1/data,/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785992227050689-9601-0/minicluster-data/ts-1/wal
uuid: "2c79466ec5a14051bbc556f5a3852645"
format_stamp: "Formatted at 2026-08-06 05:01:10 on dist-test-slave-x4qh"
server_key: "9e64761b3a121e6ad69b6aa80e61efae"
server_key_iv: "7a1c3766778b4209965db7e3fa55f2ef"
server_key_version: "encryptionkey@0"
I20260806 05:01:55.955338 2564 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785992227050689-9601-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785992227050689-9601-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785992227050689-9601-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 05:01:55.971988 2564 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 05:01:55.976282 2564 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 05:01:55.976591 2564 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 05:01:55.977627 2564 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 05:01:55.979013 2564 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 05:01:55.979106 2564 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 05:01:55.979272 2564 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 05:01:55.979363 2564 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 05:01:56.014593 2564 rpc_server.cc:307] RPC server started. Bound to: 127.9.96.66:36483
I20260806 05:01:56.014684 2755 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.96.66:36483 every 8 connection(s)
I20260806 05:01:56.016144 2564 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785992227050689-9601-0/minicluster-data/ts-1/data/info.pb
I20260806 05:01:56.018342 9601 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu as pid 2564
I20260806 05:01:56.019021 9601 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
/tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785992227050689-9601-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785992227050689-9601-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785992227050689-9601-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785992227050689-9601-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.96.67:41539
--local_ip_for_outbound_sockets=127.9.96.67
--tserver_master_addrs=127.9.96.126:34193
--webserver_port=33175
--webserver_interface=127.9.96.67
--keytab_file=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.67.keytab
--principal=kudu/127.9.96.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.96.84:41431
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785992227050689-9601-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785992227050689-9601-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785992227050689-9601-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 05:01:56.021472 2758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:01:56.019433 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:34193 (local address 127.9.96.66:55593)
0806 05:01:56.020318 (+ 885us) negotiation.cc:107] Waiting for socket to connect
0806 05:01:56.020575 (+ 257us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.96.126:34193: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":573,"thread_start_us":368,"threads_started":1}
W20260806 05:01:56.022187 2756 heartbeater.cc:644] Failed to heartbeat to 127.9.96.126:34193 (0 consecutive failures): Network error: Failed to ping master at 127.9.96.126:34193: Client connection negotiation failed: client connection to 127.9.96.126:34193: connect: Connection refused (error 111)
I20260806 05:01:56.023087 2758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:01:56.022767 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:34193 (local address 127.9.96.66:57757)
0806 05:01:56.022917 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0806 05:01:56.022982 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.96.126:34193: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260806 05:01:56.024000 2758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:01:56.023642 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:34193 (local address 127.9.96.66:49155)
0806 05:01:56.023818 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0806 05:01:56.023899 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.96.126:34193: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":83}
W20260806 05:01:56.024209 2756 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260806 05:01:56.354102 2759 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 05:01:56.354379 2759 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 05:01:56.354475 2759 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 05:01:56.361775 2759 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 05:01:56.361883 2759 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 05:01:56.362118 2759 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.96.67
I20260806 05:01:56.371658 2759 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.96.84:41431
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785992227050689-9601-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785992227050689-9601-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.67.keytab
--principal=kudu/127.9.96.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.96.67:41539
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785992227050689-9601-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.96.67
--webserver_port=33175
--tserver_master_addrs=127.9.96.126:34193
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.96.67
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785992227050689-9601-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 03:56:12 UTC on ccycloud.quasar-guijbm.root.comops.site
build id 4569618
ASAN enabled
I20260806 05:01:56.372848 2759 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 05:01:56.374120 2759 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 05:01:56.387158 2764 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 05:01:56.387085 2769 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 05:01:56.387121 2765 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 05:01:56.388442 2759 server_base.cc:1034] running on GCE node
I20260806 05:01:56.389119 2759 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 05:01:56.390033 2759 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 05:01:56.391269 2759 hybrid_clock.cc:648] HybridClock initialized: now 1785992516391229 us; error 53 us; skew 500 ppm
I20260806 05:01:56.391608 2759 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 05:01:56 dist-test-slave-x4qh krb5kdc[593](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992516, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.96.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 05:01:56.394652 2759 init.cc:377] Logged in from keytab as kudu/127.9.96.67@KRBTEST.COM (short username kudu)
I20260806 05:01:56.396174 2759 webserver.cc:460] Webserver started at http://127.9.96.67:33175/ using document root <none> and password file <none>
I20260806 05:01:56.396914 2759 fs_manager.cc:359] Metadata directory not provided
I20260806 05:01:56.397066 2759 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 05:01:56.402211 2759 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260806 05:01:56.404917 2782 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 05:01:56.406169 2759 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260806 05:01:56.406355 2759 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785992227050689-9601-0/minicluster-data/ts-2/data,/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785992227050689-9601-0/minicluster-data/ts-2/wal
uuid: "6ebbb98128aa4bd69728676853db94a2"
format_stamp: "Formatted at 2026-08-06 05:01:10 on dist-test-slave-x4qh"
server_key: "696c825852d11aca183a8bfa31ff49da"
server_key_iv: "008ddaa8c2b7b319919f9c88a012add6"
server_key_version: "encryptionkey@0"
I20260806 05:01:56.407429 2759 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785992227050689-9601-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785992227050689-9601-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785992227050689-9601-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 05:01:56.427009 2759 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 05:01:56.431689 2759 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 05:01:56.431977 2759 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 05:01:56.432989 2759 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 05:01:56.434463 2759 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 05:01:56.434597 2759 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 05:01:56.434726 2759 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 05:01:56.434818 2759 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 05:01:56.469867 2759 rpc_server.cc:307] RPC server started. Bound to: 127.9.96.67:41539
I20260806 05:01:56.469957 2946 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.96.67:41539 every 8 connection(s)
I20260806 05:01:56.471351 2759 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785992227050689-9601-0/minicluster-data/ts-2/data/info.pb
I20260806 05:01:56.476117 2952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:01:56.474273 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:34193 (local address 127.9.96.67:37965)
0806 05:01:56.474993 (+ 720us) negotiation.cc:107] Waiting for socket to connect
0806 05:01:56.475226 (+ 233us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.96.126:34193: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":481,"thread_start_us":282,"threads_started":1}
W20260806 05:01:56.476764 2949 heartbeater.cc:644] Failed to heartbeat to 127.9.96.126:34193 (0 consecutive failures): Network error: Failed to ping master at 127.9.96.126:34193: Client connection negotiation failed: client connection to 127.9.96.126:34193: connect: Connection refused (error 111)
I20260806 05:01:56.477511 2952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:01:56.477215 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:34193 (local address 127.9.96.67:57243)
0806 05:01:56.477347 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0806 05:01:56.477412 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.96.126:34193: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260806 05:01:56.478286 2952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:01:56.477999 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:34193 (local address 127.9.96.67:56995)
0806 05:01:56.478115 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0806 05:01:56.478191 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.96.126:34193: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260806 05:01:56.478520 2949 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260806 05:01:56.478545 9601 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu as pid 2759
I20260806 05:01:56.558183 2958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:01:56.557315 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:34193 (local address 127.9.96.65:52095)
0806 05:01:56.557865 (+ 550us) negotiation.cc:107] Waiting for socket to connect
0806 05:01:56.557953 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.96.126:34193: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":448,"thread_start_us":283,"threads_started":1}
I20260806 05:01:57.025605 2959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:01:57.024793 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:34193 (local address 127.9.96.66:56965)
0806 05:01:57.025263 (+ 470us) negotiation.cc:107] Waiting for socket to connect
0806 05:01:57.025377 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.96.126:34193: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":367,"thread_start_us":240,"threads_started":1}
I20260806 05:01:57.479897 2960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:01:57.479004 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:34193 (local address 127.9.96.67:59617)
0806 05:01:57.479527 (+ 523us) negotiation.cc:107] Waiting for socket to connect
0806 05:01:57.479615 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.96.126:34193: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":411,"thread_start_us":233,"threads_started":1}
I20260806 05:01:57.559846 2961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:01:57.559113 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:34193 (local address 127.9.96.65:59367)
0806 05:01:57.559601 (+ 488us) negotiation.cc:107] Waiting for socket to connect
0806 05:01:57.559684 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.96.126:34193: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":393,"thread_start_us":290,"threads_started":1}
I20260806 05:01:58.027338 2962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:01:58.026600 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:34193 (local address 127.9.96.66:43913)
0806 05:01:58.027078 (+ 478us) negotiation.cc:107] Waiting for socket to connect
0806 05:01:58.027162 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.96.126:34193: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":380,"thread_start_us":248,"threads_started":1}
I20260806 05:01:58.481441 2963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:01:58.480775 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:34193 (local address 127.9.96.67:37957)
0806 05:01:58.481208 (+ 433us) negotiation.cc:107] Waiting for socket to connect
0806 05:01:58.481283 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.96.126:34193: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":346,"thread_start_us":216,"threads_started":1}
I20260806 05:01:58.561568 2964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:01:58.560723 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:34193 (local address 127.9.96.65:35519)
0806 05:01:58.561271 (+ 548us) negotiation.cc:107] Waiting for socket to connect
0806 05:01:58.561366 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.96.126:34193: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":390,"thread_start_us":249,"threads_started":1}
I20260806 05:01:59.029016 2965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:01:59.028233 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:34193 (local address 127.9.96.66:49333)
0806 05:01:59.028690 (+ 457us) negotiation.cc:107] Waiting for socket to connect
0806 05:01:59.028822 (+ 132us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.96.126:34193: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":361,"thread_start_us":249,"threads_started":1}
I20260806 05:01:59.482836 2966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:01:59.482198 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:34193 (local address 127.9.96.67:38731)
0806 05:01:59.482590 (+ 392us) negotiation.cc:107] Waiting for socket to connect
0806 05:01:59.482664 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.96.126:34193: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":307,"thread_start_us":208,"threads_started":1}
I20260806 05:01:59.563289 2967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:01:59.562533 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:34193 (local address 127.9.96.65:60173)
0806 05:01:59.563025 (+ 492us) negotiation.cc:107] Waiting for socket to connect
0806 05:01:59.563108 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.96.126:34193: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":386,"thread_start_us":263,"threads_started":1}
I20260806 05:02:00.030897 2968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:02:00.029958 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:34193 (local address 127.9.96.66:46379)
0806 05:02:00.030512 (+ 554us) negotiation.cc:107] Waiting for socket to connect
0806 05:02:00.030623 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.96.126:34193: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":393,"thread_start_us":273,"threads_started":1}
I20260806 05:02:00.484396 2969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:02:00.483693 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:34193 (local address 127.9.96.67:35251)
0806 05:02:00.484099 (+ 406us) negotiation.cc:107] Waiting for socket to connect
0806 05:02:00.484177 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.96.126:34193: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":315,"thread_start_us":202,"threads_started":1}
I20260806 05:02:00.565007 2971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:02:00.564267 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:34193 (local address 127.9.96.65:33623)
0806 05:02:00.564741 (+ 474us) negotiation.cc:107] Waiting for socket to connect
0806 05:02:00.564821 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.96.126:34193: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":349,"thread_start_us":236,"threads_started":1}
I20260806 05:02:01.032706 2972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:02:01.031945 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:34193 (local address 127.9.96.66:33753)
0806 05:02:01.032440 (+ 495us) negotiation.cc:107] Waiting for socket to connect
0806 05:02:01.032520 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.96.126:34193: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":397,"thread_start_us":263,"threads_started":1}
I20260806 05:02:01.486075 2974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:02:01.485352 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:34193 (local address 127.9.96.67:42311)
0806 05:02:01.485840 (+ 488us) negotiation.cc:107] Waiting for socket to connect
0806 05:02:01.485916 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.96.126:34193: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":395,"thread_start_us":228,"threads_started":1}
I20260806 05:02:01.566612 2975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:02:01.565922 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:34193 (local address 127.9.96.65:56925)
0806 05:02:01.566367 (+ 445us) negotiation.cc:107] Waiting for socket to connect
0806 05:02:01.566451 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.96.126:34193: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":352,"thread_start_us":237,"threads_started":1}
I20260806 05:02:02.034282 2976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:02:02.033562 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:34193 (local address 127.9.96.66:43319)
0806 05:02:02.034029 (+ 467us) negotiation.cc:107] Waiting for socket to connect
0806 05:02:02.034114 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.96.126:34193: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":374,"thread_start_us":256,"threads_started":1}
I20260806 05:02:02.487748 2977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:02:02.486996 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:34193 (local address 127.9.96.67:52611)
0806 05:02:02.487490 (+ 494us) negotiation.cc:107] Waiting for socket to connect
0806 05:02:02.487581 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.96.126:34193: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":347,"thread_start_us":231,"threads_started":1}
I20260806 05:02:02.568465 2978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:02:02.567629 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:34193 (local address 127.9.96.65:53519)
0806 05:02:02.568175 (+ 546us) negotiation.cc:107] Waiting for socket to connect
0806 05:02:02.568257 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.96.126:34193: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":440,"thread_start_us":273,"threads_started":1}
I20260806 05:02:03.036540 2981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:02:03.035460 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:34193 (local address 127.9.96.66:43707)
0806 05:02:03.036095 (+ 635us) negotiation.cc:107] Waiting for socket to connect
0806 05:02:03.036202 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.96.126:34193: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":517,"thread_start_us":298,"threads_started":1}
I20260806 05:02:03.489406 2982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:02:03.488628 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:34193 (local address 127.9.96.67:51773)
0806 05:02:03.489117 (+ 489us) negotiation.cc:107] Waiting for socket to connect
0806 05:02:03.489195 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.96.126:34193: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":389,"thread_start_us":228,"threads_started":1}
I20260806 05:02:03.570132 2983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:02:03.569387 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:34193 (local address 127.9.96.65:41363)
0806 05:02:03.569867 (+ 480us) negotiation.cc:107] Waiting for socket to connect
0806 05:02:03.569952 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.96.126:34193: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":378,"thread_start_us":257,"threads_started":1}
I20260806 05:02:04.038571 2984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:02:04.037637 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:34193 (local address 127.9.96.66:39669)
0806 05:02:04.038249 (+ 612us) negotiation.cc:107] Waiting for socket to connect
0806 05:02:04.038355 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.96.126:34193: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":483,"thread_start_us":341,"threads_started":1}
I20260806 05:02:04.491077 2985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:02:04.490345 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:34193 (local address 127.9.96.67:49935)
0806 05:02:04.490843 (+ 498us) negotiation.cc:107] Waiting for socket to connect
0806 05:02:04.490919 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.96.126:34193: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":405,"thread_start_us":288,"threads_started":1}
I20260806 05:02:04.571877 2986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:02:04.571038 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:34193 (local address 127.9.96.65:44141)
0806 05:02:04.571579 (+ 541us) negotiation.cc:107] Waiting for socket to connect
0806 05:02:04.571692 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.96.126:34193: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":435,"thread_start_us":246,"threads_started":1}
I20260806 05:02:05.040385 2988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:02:05.039576 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:34193 (local address 127.9.96.66:49549)
0806 05:02:05.040114 (+ 538us) negotiation.cc:107] Waiting for socket to connect
0806 05:02:05.040203 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.96.126:34193: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":430,"thread_start_us":296,"threads_started":1}
I20260806 05:02:05.492741 2989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:02:05.492016 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:34193 (local address 127.9.96.67:43673)
0806 05:02:05.492493 (+ 477us) negotiation.cc:107] Waiting for socket to connect
0806 05:02:05.492573 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.96.126:34193: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":374,"thread_start_us":232,"threads_started":1}
I20260806 05:02:05.573593 2991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:02:05.572900 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:34193 (local address 127.9.96.65:58417)
0806 05:02:05.573345 (+ 445us) negotiation.cc:107] Waiting for socket to connect
0806 05:02:05.573428 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.96.126:34193: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":346,"thread_start_us":234,"threads_started":1}
I20260806 05:02:06.042215 2992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:02:06.041374 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:34193 (local address 127.9.96.66:59083)
0806 05:02:06.041966 (+ 592us) negotiation.cc:107] Waiting for socket to connect
0806 05:02:06.042047 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.96.126:34193: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":458,"thread_start_us":280,"threads_started":1}
I20260806 05:02:06.494346 2994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:02:06.493610 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:34193 (local address 127.9.96.67:37625)
0806 05:02:06.494086 (+ 476us) negotiation.cc:107] Waiting for socket to connect
0806 05:02:06.494169 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.96.126:34193: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":380,"thread_start_us":260,"threads_started":1}
I20260806 05:02:06.575304 2995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:02:06.574524 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:34193 (local address 127.9.96.65:45869)
0806 05:02:06.575036 (+ 512us) negotiation.cc:107] Waiting for socket to connect
0806 05:02:06.575117 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.96.126:34193: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":416,"thread_start_us":308,"threads_started":1}
I20260806 05:02:07.044006 2999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:02:07.043138 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:34193 (local address 127.9.96.66:51993)
0806 05:02:07.043675 (+ 537us) negotiation.cc:107] Waiting for socket to connect
0806 05:02:07.043812 (+ 137us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.96.126:34193: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":427,"thread_start_us":308,"threads_started":1}
I20260806 05:02:07.496356 3000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:02:07.495426 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:34193 (local address 127.9.96.67:60921)
0806 05:02:07.495987 (+ 561us) negotiation.cc:107] Waiting for socket to connect
0806 05:02:07.496097 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.96.126:34193: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":444,"thread_start_us":271,"threads_started":1}
I20260806 05:02:07.577097 3001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:02:07.576347 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:34193 (local address 127.9.96.65:41409)
0806 05:02:07.576839 (+ 492us) negotiation.cc:107] Waiting for socket to connect
0806 05:02:07.576932 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.96.126:34193: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":370,"thread_start_us":251,"threads_started":1}
I20260806 05:02:08.045591 3003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:02:08.044881 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:34193 (local address 127.9.96.66:37321)
0806 05:02:08.045351 (+ 470us) negotiation.cc:107] Waiting for socket to connect
0806 05:02:08.045432 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.96.126:34193: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":363,"thread_start_us":235,"threads_started":1}
I20260806 05:02:08.497946 3005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:02:08.497251 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:34193 (local address 127.9.96.67:48345)
0806 05:02:08.497705 (+ 454us) negotiation.cc:107] Waiting for socket to connect
0806 05:02:08.497789 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.96.126:34193: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":342,"thread_start_us":219,"threads_started":1}
I20260806 05:02:08.578835 3006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:02:08.578077 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:34193 (local address 127.9.96.65:57079)
0806 05:02:08.578514 (+ 437us) negotiation.cc:107] Waiting for socket to connect
0806 05:02:08.578610 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.96.126:34193: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":348,"thread_start_us":226,"threads_started":1}
I20260806 05:02:09.047197 3007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:02:09.046463 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:34193 (local address 127.9.96.66:46977)
0806 05:02:09.046922 (+ 459us) negotiation.cc:107] Waiting for socket to connect
0806 05:02:09.047015 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.96.126:34193: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":366,"thread_start_us":228,"threads_started":1}
I20260806 05:02:09.499639 3008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:02:09.498841 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:34193 (local address 127.9.96.67:60985)
0806 05:02:09.499293 (+ 452us) negotiation.cc:107] Waiting for socket to connect
0806 05:02:09.499451 (+ 158us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.96.126:34193: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":361,"thread_start_us":219,"threads_started":1}
I20260806 05:02:09.580644 3009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:02:09.579919 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:34193 (local address 127.9.96.65:37803)
0806 05:02:09.580377 (+ 458us) negotiation.cc:107] Waiting for socket to connect
0806 05:02:09.580460 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.96.126:34193: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":358,"thread_start_us":232,"threads_started":1}
I20260806 05:02:10.048830 3011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:02:10.048096 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:34193 (local address 127.9.96.66:51823)
0806 05:02:10.048548 (+ 452us) negotiation.cc:107] Waiting for socket to connect
0806 05:02:10.048627 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.96.126:34193: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":342,"thread_start_us":237,"threads_started":1}
I20260806 05:02:10.501147 3013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:02:10.500424 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:34193 (local address 127.9.96.67:44995)
0806 05:02:10.500886 (+ 462us) negotiation.cc:107] Waiting for socket to connect
0806 05:02:10.500969 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.96.126:34193: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":361,"thread_start_us":241,"threads_started":1}
I20260806 05:02:10.582708 3014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:02:10.581887 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:34193 (local address 127.9.96.65:47747)
0806 05:02:10.582432 (+ 545us) negotiation.cc:107] Waiting for socket to connect
0806 05:02:10.582519 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.96.126:34193: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":444,"thread_start_us":333,"threads_started":1}
I20260806 05:02:11.050681 3015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:02:11.049947 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:34193 (local address 127.9.96.66:48033)
0806 05:02:11.050381 (+ 434us) negotiation.cc:107] Waiting for socket to connect
0806 05:02:11.050479 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.96.126:34193: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":335,"thread_start_us":229,"threads_started":1}
I20260806 05:02:11.487296 9601 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu pid 2298
I20260806 05:02:11.503258 3017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:02:11.502362 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:34193 (local address 127.9.96.67:45179)
0806 05:02:11.502837 (+ 475us) negotiation.cc:107] Waiting for socket to connect
0806 05:02:11.502993 (+ 156us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.96.126:34193: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":368,"thread_start_us":240,"threads_started":1}
I20260806 05:02:11.506536 3016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:02:11.488520 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49110 (local address 127.9.96.65:45547)
0806 05:02:11.489047 (+ 527us) server_negotiation.cc:206] Beginning negotiation
0806 05:02:11.489060 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 05:02:11.489132 (+ 72us) server_negotiation.cc:407] Connection header received
0806 05:02:11.490886 (+ 1754us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 05:02:11.490895 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 05:02:11.491317 (+ 422us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 05:02:11.491682 (+ 365us) server_negotiation.cc:226] Negotiated authn=SASL
0806 05:02:11.493064 (+ 1382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:02:11.494393 (+ 1329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:02:11.495808 (+ 1415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:02:11.496474 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:02:11.496608 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:02:11.499454 (+ 2846us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 05:02:11.499464 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 05:02:11.499479 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 05:02:11.499526 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0806 05:02:11.502834 (+ 3308us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 05:02:11.503825 (+ 991us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 05:02:11.503832 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 05:02:11.503839 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 05:02:11.504008 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 05:02:11.504357 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 05:02:11.504361 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 05:02:11.504363 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 05:02:11.504919 (+ 556us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 05:02:11.505155 (+ 236us) server_negotiation.cc:1035] Waiting for connection context
0806 05:02:11.505514 (+ 359us) server_negotiation.cc:299] Negotiation successful
0806 05:02:11.506007 (+ 493us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":381,"thread_start_us":259,"threads_started":1}
I20260806 05:02:11.508488 2546 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 05:02:11.605726 9601 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu with pid 2298
I20260806 05:02:11.605813 3019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:02:11.604658 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:34193 (local address 127.9.96.65:42297)
0806 05:02:11.605272 (+ 614us) negotiation.cc:107] Waiting for socket to connect
0806 05:02:11.605430 (+ 158us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.96.126:34193: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":475,"thread_start_us":330,"threads_started":1}
I20260806 05:02:11.630602 9601 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu pid 2564
I20260806 05:02:11.649482 3020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:02:11.631676 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52876 (local address 127.9.96.66:36483)
0806 05:02:11.632272 (+ 596us) server_negotiation.cc:206] Beginning negotiation
0806 05:02:11.632287 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 05:02:11.632327 (+ 40us) server_negotiation.cc:407] Connection header received
0806 05:02:11.633933 (+ 1606us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 05:02:11.633943 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 05:02:11.634378 (+ 435us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 05:02:11.634672 (+ 294us) server_negotiation.cc:226] Negotiated authn=SASL
0806 05:02:11.636500 (+ 1828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:02:11.637801 (+ 1301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:02:11.639186 (+ 1385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:02:11.639944 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:02:11.640078 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:02:11.643099 (+ 3021us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 05:02:11.643107 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 05:02:11.643122 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 05:02:11.643167 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0806 05:02:11.646305 (+ 3138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 05:02:11.646977 (+ 672us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 05:02:11.646984 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 05:02:11.646990 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 05:02:11.647167 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 05:02:11.647588 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 05:02:11.647591 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 05:02:11.647593 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 05:02:11.648158 (+ 565us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 05:02:11.648397 (+ 239us) server_negotiation.cc:1035] Waiting for connection context
0806 05:02:11.648868 (+ 471us) server_negotiation.cc:299] Negotiation successful
0806 05:02:11.649185 (+ 317us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":420,"thread_start_us":302,"threads_started":1}
I20260806 05:02:11.650954 2749 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 05:02:11.734400 9601 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu with pid 2564
I20260806 05:02:11.759843 9601 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu pid 2759
I20260806 05:02:11.778093 3024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:02:11.760999 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52602 (local address 127.9.96.67:41539)
0806 05:02:11.761694 (+ 695us) server_negotiation.cc:206] Beginning negotiation
0806 05:02:11.761708 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0806 05:02:11.761781 (+ 73us) server_negotiation.cc:407] Connection header received
0806 05:02:11.763658 (+ 1877us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 05:02:11.763670 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 05:02:11.764142 (+ 472us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 05:02:11.764445 (+ 303us) server_negotiation.cc:226] Negotiated authn=SASL
0806 05:02:11.765872 (+ 1427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:02:11.767145 (+ 1273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:02:11.768615 (+ 1470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:02:11.769313 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:02:11.769456 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:02:11.772124 (+ 2668us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 05:02:11.772132 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 05:02:11.772148 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 05:02:11.772186 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0806 05:02:11.775202 (+ 3016us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 05:02:11.775888 (+ 686us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 05:02:11.775895 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 05:02:11.775902 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 05:02:11.776073 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 05:02:11.776395 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 05:02:11.776398 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 05:02:11.776399 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 05:02:11.776908 (+ 509us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 05:02:11.777140 (+ 232us) server_negotiation.cc:1035] Waiting for connection context
0806 05:02:11.777438 (+ 298us) server_negotiation.cc:299] Negotiation successful
0806 05:02:11.777787 (+ 349us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":530,"thread_start_us":357,"threads_started":1}
I20260806 05:02:11.779627 2940 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 05:02:11.861827 9601 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu with pid 2759
2026-08-06T05:02:11Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (64924 ms)
[----------] 30 tests from SecurityITest (304738 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785992227050689-9601-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 05:02:11 dist-test-slave-x4qh krb5kdc[3029](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 05:02:11 dist-test-slave-x4qh krb5kdc[3029](info): set up 2 sockets
Aug 06 05:02:11 dist-test-slave-x4qh krb5kdc[3029](info): commencing operation
krb5kdc: starting...
W20260806 05:02:13.193467 9601 mini_kdc.cc:121] Time spent starting KDC: real 1.283s user 0.000s sys 0.039s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785992227050689-9601-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785992227050689-9601-0/krb5kdc/test-user.keytab.
Aug 06 05:02:13 dist-test-slave-x4qh krb5kdc[3029](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992533, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T05:02:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T05:02:13Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.96.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.96.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.96.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.126.keytab.
Entry for principal kudu/127.9.96.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.126.keytab.
I20260806 05:02:13.486701 9601 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
/tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785992227050689-9601-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785992227050689-9601-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785992227050689-9601-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785992227050689-9601-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.96.126:46585
--webserver_interface=127.9.96.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.126.keytab
--principal=kudu/127.9.96.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.96.84:42229
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.96.126:46585
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785992227050689-9601-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785992227050689-9601-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785992227050689-9601-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 05:02:13.834833 3045 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 05:02:13.835249 3045 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 05:02:13.835498 3045 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 05:02:13.842907 3045 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 05:02:13.843070 3045 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 05:02:13.843199 3045 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 05:02:13.843317 3045 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 05:02:13.843436 3045 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 05:02:13.853717 3045 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.96.84:42229
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785992227050689-9601-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785992227050689-9601-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.96.126:46585
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.126.keytab
--principal=kudu/127.9.96.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.96.126:46585
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785992227050689-9601-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.96.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785992227050689-9601-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 03:56:12 UTC on ccycloud.quasar-guijbm.root.comops.site
build id 4569618
ASAN enabled
I20260806 05:02:13.855086 3045 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 05:02:13.856518 3045 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 05:02:13.869511 3054 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 05:02:13.869448 3059 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 05:02:13.869563 3061 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 05:02:13.869707 3045 server_base.cc:1034] running on GCE node
I20260806 05:02:13.871047 3045 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 05:02:13.872534 3045 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 05:02:13.873754 3045 hybrid_clock.cc:648] HybridClock initialized: now 1785992533873679 us; error 53 us; skew 500 ppm
I20260806 05:02:13.874114 3045 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 05:02:13 dist-test-slave-x4qh krb5kdc[3029](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992533, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.96.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 05:02:13.881011 3045 init.cc:377] Logged in from keytab as kudu/127.9.96.126@KRBTEST.COM (short username kudu)
I20260806 05:02:13.882606 3045 webserver.cc:460] Webserver started at http://127.9.96.126:40051/ using document root <none> and password file <none>
I20260806 05:02:13.883312 3045 fs_manager.cc:359] Metadata directory not provided
I20260806 05:02:13.883502 3045 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 05:02:13.883843 3045 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 05:02:13.887014 3045 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785992227050689-9601-0/minicluster-data/master-0/data/instance:
uuid: "73e62ec4a04a4fd38a4e2724ac34dd1a"
format_stamp: "Formatted at 2026-08-06 05:02:13 on dist-test-slave-x4qh"
server_key: "10a0d5da57a91fe550b0864eb6d08737"
server_key_iv: "1bd6f78e38b11045bbc629c27ca71876"
server_key_version: "encryptionkey@0"
I20260806 05:02:13.888001 3045 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785992227050689-9601-0/minicluster-data/master-0/wal/instance:
uuid: "73e62ec4a04a4fd38a4e2724ac34dd1a"
format_stamp: "Formatted at 2026-08-06 05:02:13 on dist-test-slave-x4qh"
server_key: "10a0d5da57a91fe550b0864eb6d08737"
server_key_iv: "1bd6f78e38b11045bbc629c27ca71876"
server_key_version: "encryptionkey@0"
I20260806 05:02:13.893440 3045 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260806 05:02:13.897126 3076 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 05:02:13.898435 3045 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260806 05:02:13.898640 3045 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785992227050689-9601-0/minicluster-data/master-0/data,/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785992227050689-9601-0/minicluster-data/master-0/wal
uuid: "73e62ec4a04a4fd38a4e2724ac34dd1a"
format_stamp: "Formatted at 2026-08-06 05:02:13 on dist-test-slave-x4qh"
server_key: "10a0d5da57a91fe550b0864eb6d08737"
server_key_iv: "1bd6f78e38b11045bbc629c27ca71876"
server_key_version: "encryptionkey@0"
I20260806 05:02:13.898837 3045 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785992227050689-9601-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785992227050689-9601-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785992227050689-9601-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 05:02:13.938010 3045 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 05:02:13.942106 3045 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 05:02:13.942420 3045 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 05:02:13.962529 3045 rpc_server.cc:307] RPC server started. Bound to: 127.9.96.126:46585
I20260806 05:02:13.962574 3157 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.96.126:46585 every 8 connection(s)
I20260806 05:02:13.963977 3045 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785992227050689-9601-0/minicluster-data/master-0/data/info.pb
I20260806 05:02:13.966245 9601 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu as pid 3045
I20260806 05:02:13.966490 9601 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785992227050689-9601-0/minicluster-data/master-0/wal/instance
I20260806 05:02:13.967185 9601 external_mini_cluster.cc:1428] Setting key 3a8afff07d8335cf7a9aac649cfaad1d
I20260806 05:02:13.967975 3158 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 05:02:13.982589 3158 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 73e62ec4a04a4fd38a4e2724ac34dd1a: Bootstrap starting.
Aug 06 05:02:13 dist-test-slave-x4qh krb5kdc[3029](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992533, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.96.126@KRBTEST.COM
I20260806 05:02:13.986608 3158 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 73e62ec4a04a4fd38a4e2724ac34dd1a: Neither blocks nor log segments found. Creating new log.
I20260806 05:02:13.987792 3158 log.cc:824] T 00000000000000000000000000000000 P 73e62ec4a04a4fd38a4e2724ac34dd1a: Log is configured to *not* fsync() on all Append() calls
I20260806 05:02:13.990598 3158 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 73e62ec4a04a4fd38a4e2724ac34dd1a: No bootstrap required, opened a new log
I20260806 05:02:13.992393 3161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:02:13.969351 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58568 (local address 127.9.96.126:46585)
0806 05:02:13.970205 (+ 854us) server_negotiation.cc:206] Beginning negotiation
0806 05:02:13.970220 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 05:02:13.970269 (+ 49us) server_negotiation.cc:407] Connection header received
0806 05:02:13.972883 (+ 2614us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 05:02:13.972910 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 05:02:13.973405 (+ 495us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 05:02:13.973850 (+ 445us) server_negotiation.cc:226] Negotiated authn=SASL
0806 05:02:13.975246 (+ 1396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:02:13.977301 (+ 2055us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:02:13.978762 (+ 1461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:02:13.979942 (+ 1180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:02:13.980133 (+ 191us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:02:13.984871 (+ 4738us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 05:02:13.984879 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 05:02:13.984893 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 05:02:13.984931 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0806 05:02:13.988191 (+ 3260us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 05:02:13.988979 (+ 788us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 05:02:13.988986 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 05:02:13.988993 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 05:02:13.989167 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 05:02:13.989581 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 05:02:13.989586 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 05:02:13.989589 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 05:02:13.990109 (+ 520us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 05:02:13.990438 (+ 329us) server_negotiation.cc:1035] Waiting for connection context
0806 05:02:13.990936 (+ 498us) server_negotiation.cc:299] Negotiation successful
0806 05:02:13.991415 (+ 479us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":623,"thread_start_us":341,"threads_started":1}
I20260806 05:02:13.999207 3158 raft_consensus.cc:348] T 00000000000000000000000000000000 P 73e62ec4a04a4fd38a4e2724ac34dd1a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "73e62ec4a04a4fd38a4e2724ac34dd1a" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 46585 } }
I20260806 05:02:13.999665 3158 raft_consensus.cc:374] T 00000000000000000000000000000000 P 73e62ec4a04a4fd38a4e2724ac34dd1a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 05:02:13.999785 3158 raft_consensus.cc:729] T 00000000000000000000000000000000 P 73e62ec4a04a4fd38a4e2724ac34dd1a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 73e62ec4a04a4fd38a4e2724ac34dd1a, State: Initialized, Role: FOLLOWER
I20260806 05:02:14.000447 3158 consensus_queue.cc:260] T 00000000000000000000000000000000 P 73e62ec4a04a4fd38a4e2724ac34dd1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "73e62ec4a04a4fd38a4e2724ac34dd1a" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 46585 } }
I20260806 05:02:14.000761 3158 raft_consensus.cc:388] T 00000000000000000000000000000000 P 73e62ec4a04a4fd38a4e2724ac34dd1a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 05:02:14.000903 3158 raft_consensus.cc:482] T 00000000000000000000000000000000 P 73e62ec4a04a4fd38a4e2724ac34dd1a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 05:02:14.001086 3158 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 73e62ec4a04a4fd38a4e2724ac34dd1a [term 0 FOLLOWER]: Advancing to term 1
I20260806 05:02:14.003749 3158 raft_consensus.cc:504] T 00000000000000000000000000000000 P 73e62ec4a04a4fd38a4e2724ac34dd1a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "73e62ec4a04a4fd38a4e2724ac34dd1a" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 46585 } }
I20260806 05:02:14.004310 3158 leader_election.cc:302] T 00000000000000000000000000000000 P 73e62ec4a04a4fd38a4e2724ac34dd1a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 73e62ec4a04a4fd38a4e2724ac34dd1a; no voters:
I20260806 05:02:14.005061 3158 leader_election.cc:288] T 00000000000000000000000000000000 P 73e62ec4a04a4fd38a4e2724ac34dd1a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 05:02:14.005216 3164 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 73e62ec4a04a4fd38a4e2724ac34dd1a [term 1 FOLLOWER]: Leader election won for term 1
I20260806 05:02:14.005962 3164 raft_consensus.cc:686] T 00000000000000000000000000000000 P 73e62ec4a04a4fd38a4e2724ac34dd1a [term 1 LEADER]: Becoming Leader. State: Replica: 73e62ec4a04a4fd38a4e2724ac34dd1a, State: Running, Role: LEADER
I20260806 05:02:14.006520 3164 consensus_queue.cc:237] T 00000000000000000000000000000000 P 73e62ec4a04a4fd38a4e2724ac34dd1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "73e62ec4a04a4fd38a4e2724ac34dd1a" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 46585 } }
I20260806 05:02:14.007208 3158 sys_catalog.cc:564] T 00000000000000000000000000000000 P 73e62ec4a04a4fd38a4e2724ac34dd1a [sys.catalog]: configured and running, proceeding with master startup.
I20260806 05:02:14.009543 3165 sys_catalog.cc:455] T 00000000000000000000000000000000 P 73e62ec4a04a4fd38a4e2724ac34dd1a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "73e62ec4a04a4fd38a4e2724ac34dd1a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "73e62ec4a04a4fd38a4e2724ac34dd1a" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 46585 } } }
I20260806 05:02:14.009984 3165 sys_catalog.cc:458] T 00000000000000000000000000000000 P 73e62ec4a04a4fd38a4e2724ac34dd1a [sys.catalog]: This master's current role is: LEADER
I20260806 05:02:14.009542 3166 sys_catalog.cc:455] T 00000000000000000000000000000000 P 73e62ec4a04a4fd38a4e2724ac34dd1a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 73e62ec4a04a4fd38a4e2724ac34dd1a. Latest consensus state: current_term: 1 leader_uuid: "73e62ec4a04a4fd38a4e2724ac34dd1a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "73e62ec4a04a4fd38a4e2724ac34dd1a" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 46585 } } }
I20260806 05:02:14.010525 3166 sys_catalog.cc:458] T 00000000000000000000000000000000 P 73e62ec4a04a4fd38a4e2724ac34dd1a [sys.catalog]: This master's current role is: LEADER
W20260806 05:02:14.018051 3185 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 73e62ec4a04a4fd38a4e2724ac34dd1a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 05:02:14.018199 3185 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 05:02:14.018333 3173 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 05:02:14.020082 3173 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 05:02:14.028391 3173 catalog_manager.cc:1347] Generated new cluster ID: 656e4e04bc5940d9926f21ee04347e26
I20260806 05:02:14.028510 3173 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 05:02:14.048295 3173 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 05:02:14.049599 3173 catalog_manager.cc:1504] Loading token signing keys...
I20260806 05:02:14.054916 3173 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 73e62ec4a04a4fd38a4e2724ac34dd1a: Generated new TSK 0
I20260806 05:02:14.055701 3173 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.9.96.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.96.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.96.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.65.keytab.
Entry for principal kudu/127.9.96.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.65.keytab.
I20260806 05:02:14.131266 9601 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
/tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785992227050689-9601-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785992227050689-9601-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785992227050689-9601-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785992227050689-9601-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.96.65:0
--local_ip_for_outbound_sockets=127.9.96.65
--webserver_interface=127.9.96.65
--webserver_port=0
--tserver_master_addrs=127.9.96.126:46585
--keytab_file=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.65.keytab
--principal=kudu/127.9.96.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.96.84:42229
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785992227050689-9601-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785992227050689-9601-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785992227050689-9601-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 05:02:14.472887 3192 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 05:02:14.473176 3192 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 05:02:14.473270 3192 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 05:02:14.480330 3192 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 05:02:14.480427 3192 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 05:02:14.480723 3192 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.96.65
I20260806 05:02:14.494120 3192 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.96.84:42229
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785992227050689-9601-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785992227050689-9601-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.65.keytab
--principal=kudu/127.9.96.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.96.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785992227050689-9601-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.96.65
--webserver_port=0
--tserver_master_addrs=127.9.96.126:46585
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.96.65
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785992227050689-9601-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 03:56:12 UTC on ccycloud.quasar-guijbm.root.comops.site
build id 4569618
ASAN enabled
I20260806 05:02:14.495285 3192 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 05:02:14.496590 3192 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 05:02:14.510282 3198 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 05:02:14.510516 3200 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 05:02:14.510272 3202 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 05:02:14.513098 3192 server_base.cc:1034] running on GCE node
I20260806 05:02:14.513700 3192 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 05:02:14.514622 3192 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 05:02:14.515823 3192 hybrid_clock.cc:648] HybridClock initialized: now 1785992534515780 us; error 33 us; skew 500 ppm
I20260806 05:02:14.516085 3192 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 05:02:14 dist-test-slave-x4qh krb5kdc[3029](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992534, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.96.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 05:02:14.519196 3192 init.cc:377] Logged in from keytab as kudu/127.9.96.65@KRBTEST.COM (short username kudu)
I20260806 05:02:14.520620 3192 webserver.cc:460] Webserver started at http://127.9.96.65:32797/ using document root <none> and password file <none>
I20260806 05:02:14.521283 3192 fs_manager.cc:359] Metadata directory not provided
I20260806 05:02:14.521384 3192 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 05:02:14.521629 3192 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 05:02:14.524482 3192 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785992227050689-9601-0/minicluster-data/ts-0/data/instance:
uuid: "62bb147239a74aab8c7fe7f7f90a757e"
format_stamp: "Formatted at 2026-08-06 05:02:14 on dist-test-slave-x4qh"
server_key: "c5fc1a290fae052acfd91373469fd799"
server_key_iv: "15e3c871b13d089c6463650d3970b095"
server_key_version: "encryptionkey@0"
I20260806 05:02:14.525226 3192 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785992227050689-9601-0/minicluster-data/ts-0/wal/instance:
uuid: "62bb147239a74aab8c7fe7f7f90a757e"
format_stamp: "Formatted at 2026-08-06 05:02:14 on dist-test-slave-x4qh"
server_key: "c5fc1a290fae052acfd91373469fd799"
server_key_iv: "15e3c871b13d089c6463650d3970b095"
server_key_version: "encryptionkey@0"
I20260806 05:02:14.529994 3192 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260806 05:02:14.533180 3210 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 05:02:14.534560 3192 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 05:02:14.534715 3192 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785992227050689-9601-0/minicluster-data/ts-0/data,/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785992227050689-9601-0/minicluster-data/ts-0/wal
uuid: "62bb147239a74aab8c7fe7f7f90a757e"
format_stamp: "Formatted at 2026-08-06 05:02:14 on dist-test-slave-x4qh"
server_key: "c5fc1a290fae052acfd91373469fd799"
server_key_iv: "15e3c871b13d089c6463650d3970b095"
server_key_version: "encryptionkey@0"
I20260806 05:02:14.534888 3192 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785992227050689-9601-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785992227050689-9601-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785992227050689-9601-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 05:02:14.553791 3192 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 05:02:14.558509 3192 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 05:02:14.558751 3192 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 05:02:14.559653 3192 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 05:02:14.560956 3192 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 05:02:14.561035 3192 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 05:02:14.561138 3192 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 05:02:14.561194 3192 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 05:02:14.598347 3192 rpc_server.cc:307] RPC server started. Bound to: 127.9.96.65:39517
I20260806 05:02:14.598394 3367 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.96.65:39517 every 8 connection(s)
I20260806 05:02:14.599858 3192 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785992227050689-9601-0/minicluster-data/ts-0/data/info.pb
I20260806 05:02:14.610260 9601 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu as pid 3192
I20260806 05:02:14.610483 9601 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785992227050689-9601-0/minicluster-data/ts-0/wal/instance
I20260806 05:02:14.611274 9601 external_mini_cluster.cc:1428] Setting key efd6300325842f00e5f339596cb5fdb3
Aug 06 05:02:14 dist-test-slave-x4qh krb5kdc[3029](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992534, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.96.65@KRBTEST.COM for kudu/127.9.96.126@KRBTEST.COM
I20260806 05:02:14.623101 3372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:02:14.602618 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.96.65:39705 (local address 127.9.96.126:46585)
0806 05:02:14.603140 (+ 522us) server_negotiation.cc:206] Beginning negotiation
0806 05:02:14.603144 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 05:02:14.604129 (+ 985us) server_negotiation.cc:407] Connection header received
0806 05:02:14.606156 (+ 2027us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 05:02:14.606162 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 05:02:14.606270 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 05:02:14.606406 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0806 05:02:14.608126 (+ 1720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:02:14.608958 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:02:14.610894 (+ 1936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:02:14.611436 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:02:14.611592 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:02:14.615896 (+ 4304us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 05:02:14.615904 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 05:02:14.615911 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 05:02:14.615955 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0806 05:02:14.619695 (+ 3740us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 05:02:14.620499 (+ 804us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 05:02:14.620508 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 05:02:14.620514 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 05:02:14.620666 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 05:02:14.621094 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 05:02:14.621101 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 05:02:14.621107 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 05:02:14.621486 (+ 379us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 05:02:14.621645 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0806 05:02:14.622347 (+ 702us) server_negotiation.cc:299] Negotiation successful
0806 05:02:14.622597 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":406,"thread_start_us":278,"threads_started":1}
I20260806 05:02:14.623771 3374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:02:14.603177 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:46585 (local address 127.9.96.65:39705)
0806 05:02:14.603959 (+ 782us) negotiation.cc:107] Waiting for socket to connect
0806 05:02:14.604009 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0806 05:02:14.605870 (+ 1861us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 05:02:14.606660 (+ 790us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 05:02:14.606667 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 05:02:14.607203 (+ 536us) client_negotiation.cc:189] Negotiated authn=SASL
0806 05:02:14.607926 (+ 723us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 05:02:14.607944 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 05:02:14.609163 (+ 1219us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 05:02:14.609167 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 05:02:14.610726 (+ 1559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 05:02:14.610740 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 05:02:14.611716 (+ 976us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 05:02:14.611725 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 05:02:14.612062 (+ 337us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:02:14.612981 (+ 919us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 05:02:14.613019 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0806 05:02:14.615667 (+ 2648us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 05:02:14.619861 (+ 4194us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 05:02:14.619867 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 05:02:14.619879 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0806 05:02:14.620358 (+ 479us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 05:02:14.620780 (+ 422us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 05:02:14.620783 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 05:02:14.620785 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 05:02:14.620973 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 05:02:14.621615 (+ 642us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 05:02:14.621620 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 05:02:14.621936 (+ 316us) client_negotiation.cc:769] Sending connection context
0806 05:02:14.622187 (+ 251us) client_negotiation.cc:240] Negotiation successful
0806 05:02:14.622559 (+ 372us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":545,"thread_start_us":329,"threads_started":1}
I20260806 05:02:14.625353 3368 heartbeater.cc:347] Connected to a master server at 127.9.96.126:46585
I20260806 05:02:14.625671 3368 heartbeater.cc:464] Registering TS with master...
I20260806 05:02:14.626551 3368 heartbeater.cc:511] Master 127.9.96.126:46585 requested a full tablet report, sending...
I20260806 05:02:14.628662 3104 ts_manager.cc:194] Registered new tserver with Master: 62bb147239a74aab8c7fe7f7f90a757e (127.9.96.65:39517)
I20260806 05:02:14.630030 3104 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.96.65@KRBTEST.COM'} at 127.9.96.65:39705
WARNING: no policy specified for kudu/127.9.96.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.96.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.96.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.66.keytab.
Entry for principal kudu/127.9.96.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.66.keytab.
I20260806 05:02:14.678206 9601 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
/tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785992227050689-9601-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785992227050689-9601-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785992227050689-9601-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785992227050689-9601-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.96.66:0
--local_ip_for_outbound_sockets=127.9.96.66
--webserver_interface=127.9.96.66
--webserver_port=0
--tserver_master_addrs=127.9.96.126:46585
--keytab_file=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.66.keytab
--principal=kudu/127.9.96.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.96.84:42229
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785992227050689-9601-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785992227050689-9601-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785992227050689-9601-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 05:02:15.012743 3377 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 05:02:15.013034 3377 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 05:02:15.013223 3377 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 05:02:15.020475 3377 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 05:02:15.020603 3377 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 05:02:15.020788 3377 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.96.66
I20260806 05:02:15.030251 3377 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.96.84:42229
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785992227050689-9601-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785992227050689-9601-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.66.keytab
--principal=kudu/127.9.96.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.96.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785992227050689-9601-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.96.66
--webserver_port=0
--tserver_master_addrs=127.9.96.126:46585
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.96.66
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785992227050689-9601-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 03:56:12 UTC on ccycloud.quasar-guijbm.root.comops.site
build id 4569618
ASAN enabled
I20260806 05:02:15.031833 3377 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 05:02:15.033377 3377 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 05:02:15.046941 3383 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 05:02:15.046981 3385 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 05:02:15.047281 3387 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 05:02:15.048225 3377 server_base.cc:1034] running on GCE node
I20260806 05:02:15.048882 3377 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 05:02:15.049767 3377 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 05:02:15.050973 3377 hybrid_clock.cc:648] HybridClock initialized: now 1785992535050923 us; error 44 us; skew 500 ppm
I20260806 05:02:15.051287 3377 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 05:02:15 dist-test-slave-x4qh krb5kdc[3029](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992535, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.96.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 05:02:15.054514 3377 init.cc:377] Logged in from keytab as kudu/127.9.96.66@KRBTEST.COM (short username kudu)
I20260806 05:02:15.055986 3377 webserver.cc:460] Webserver started at http://127.9.96.66:45827/ using document root <none> and password file <none>
I20260806 05:02:15.056640 3377 fs_manager.cc:359] Metadata directory not provided
I20260806 05:02:15.056739 3377 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 05:02:15.056988 3377 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 05:02:15.059828 3377 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785992227050689-9601-0/minicluster-data/ts-1/data/instance:
uuid: "6b2713f209924713a4e746ced443630e"
format_stamp: "Formatted at 2026-08-06 05:02:15 on dist-test-slave-x4qh"
server_key: "d57474d62ffd078b89f1ddaed9068712"
server_key_iv: "74b24718cf61439c849524bd948c5630"
server_key_version: "encryptionkey@0"
I20260806 05:02:15.060565 3377 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785992227050689-9601-0/minicluster-data/ts-1/wal/instance:
uuid: "6b2713f209924713a4e746ced443630e"
format_stamp: "Formatted at 2026-08-06 05:02:15 on dist-test-slave-x4qh"
server_key: "d57474d62ffd078b89f1ddaed9068712"
server_key_iv: "74b24718cf61439c849524bd948c5630"
server_key_version: "encryptionkey@0"
I20260806 05:02:15.065666 3377 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260806 05:02:15.069235 3400 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 05:02:15.070677 3377 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260806 05:02:15.070904 3377 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785992227050689-9601-0/minicluster-data/ts-1/data,/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785992227050689-9601-0/minicluster-data/ts-1/wal
uuid: "6b2713f209924713a4e746ced443630e"
format_stamp: "Formatted at 2026-08-06 05:02:15 on dist-test-slave-x4qh"
server_key: "d57474d62ffd078b89f1ddaed9068712"
server_key_iv: "74b24718cf61439c849524bd948c5630"
server_key_version: "encryptionkey@0"
I20260806 05:02:15.071082 3377 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785992227050689-9601-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785992227050689-9601-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785992227050689-9601-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 05:02:15.085765 3377 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 05:02:15.090219 3377 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 05:02:15.090619 3377 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 05:02:15.091825 3377 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 05:02:15.093364 3377 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 05:02:15.093457 3377 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 05:02:15.093649 3377 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 05:02:15.093721 3377 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 05:02:15.131913 3377 rpc_server.cc:307] RPC server started. Bound to: 127.9.96.66:42971
I20260806 05:02:15.132033 3565 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.96.66:42971 every 8 connection(s)
I20260806 05:02:15.133177 3377 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785992227050689-9601-0/minicluster-data/ts-1/data/info.pb
I20260806 05:02:15.136818 9601 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu as pid 3377
I20260806 05:02:15.137002 9601 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785992227050689-9601-0/minicluster-data/ts-1/wal/instance
I20260806 05:02:15.137545 9601 external_mini_cluster.cc:1428] Setting key ff5e5efc05d72da1a3dbf784f32cad38
Aug 06 05:02:15 dist-test-slave-x4qh krb5kdc[3029](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992535, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.96.66@KRBTEST.COM for kudu/127.9.96.126@KRBTEST.COM
I20260806 05:02:15.153268 3573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:02:15.135697 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.96.66:49949 (local address 127.9.96.126:46585)
0806 05:02:15.136097 (+ 400us) server_negotiation.cc:206] Beginning negotiation
0806 05:02:15.136102 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 05:02:15.137238 (+ 1136us) server_negotiation.cc:407] Connection header received
0806 05:02:15.139493 (+ 2255us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 05:02:15.139499 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 05:02:15.139589 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 05:02:15.139708 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0806 05:02:15.141152 (+ 1444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:02:15.141868 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:02:15.143440 (+ 1572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:02:15.143855 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:02:15.143991 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:02:15.147448 (+ 3457us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 05:02:15.147453 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 05:02:15.147456 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 05:02:15.147487 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0806 05:02:15.150082 (+ 2595us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 05:02:15.150909 (+ 827us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 05:02:15.150918 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 05:02:15.150921 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 05:02:15.151050 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 05:02:15.151538 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 05:02:15.151544 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 05:02:15.151547 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 05:02:15.151823 (+ 276us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 05:02:15.152016 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0806 05:02:15.152689 (+ 673us) server_negotiation.cc:299] Negotiation successful
0806 05:02:15.152920 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":298,"thread_start_us":198,"threads_started":1}
I20260806 05:02:15.154192 3574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:02:15.136240 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:46585 (local address 127.9.96.66:49949)
0806 05:02:15.137026 (+ 786us) negotiation.cc:107] Waiting for socket to connect
0806 05:02:15.137112 (+ 86us) client_negotiation.cc:174] Beginning negotiation
0806 05:02:15.139163 (+ 2051us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 05:02:15.139935 (+ 772us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 05:02:15.139943 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 05:02:15.140452 (+ 509us) client_negotiation.cc:189] Negotiated authn=SASL
0806 05:02:15.140987 (+ 535us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 05:02:15.141003 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 05:02:15.142022 (+ 1019us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 05:02:15.142030 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 05:02:15.143238 (+ 1208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 05:02:15.143263 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 05:02:15.144089 (+ 826us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 05:02:15.144096 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 05:02:15.144320 (+ 224us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:02:15.145018 (+ 698us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 05:02:15.145039 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0806 05:02:15.147170 (+ 2131us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 05:02:15.150241 (+ 3071us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 05:02:15.150250 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 05:02:15.150264 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0806 05:02:15.150789 (+ 525us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 05:02:15.151166 (+ 377us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 05:02:15.151171 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 05:02:15.151175 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 05:02:15.151415 (+ 240us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 05:02:15.151955 (+ 540us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 05:02:15.151965 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 05:02:15.152333 (+ 368us) client_negotiation.cc:769] Sending connection context
0806 05:02:15.152584 (+ 251us) client_negotiation.cc:240] Negotiation successful
0806 05:02:15.153075 (+ 491us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":521,"thread_start_us":345,"threads_started":1}
I20260806 05:02:15.155679 3566 heartbeater.cc:347] Connected to a master server at 127.9.96.126:46585
I20260806 05:02:15.156036 3566 heartbeater.cc:464] Registering TS with master...
I20260806 05:02:15.156893 3566 heartbeater.cc:511] Master 127.9.96.126:46585 requested a full tablet report, sending...
I20260806 05:02:15.158550 3104 ts_manager.cc:194] Registered new tserver with Master: 6b2713f209924713a4e746ced443630e (127.9.96.66:42971)
I20260806 05:02:15.159551 3104 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.96.66@KRBTEST.COM'} at 127.9.96.66:49949
WARNING: no policy specified for kudu/127.9.96.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.96.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.96.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.67.keytab.
Entry for principal kudu/127.9.96.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.67.keytab.
I20260806 05:02:15.206548 9601 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
/tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785992227050689-9601-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785992227050689-9601-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785992227050689-9601-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785992227050689-9601-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.96.67:0
--local_ip_for_outbound_sockets=127.9.96.67
--webserver_interface=127.9.96.67
--webserver_port=0
--tserver_master_addrs=127.9.96.126:46585
--keytab_file=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.67.keytab
--principal=kudu/127.9.96.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.96.84:42229
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785992227050689-9601-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785992227050689-9601-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785992227050689-9601-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 05:02:15.550010 3578 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 05:02:15.550283 3578 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 05:02:15.550377 3578 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 05:02:15.558035 3578 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 05:02:15.558140 3578 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 05:02:15.558252 3578 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.96.67
I20260806 05:02:15.567947 3578 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.96.84:42229
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785992227050689-9601-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785992227050689-9601-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.67.keytab
--principal=kudu/127.9.96.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.96.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785992227050689-9601-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.96.67
--webserver_port=0
--tserver_master_addrs=127.9.96.126:46585
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.96.67
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785992227050689-9601-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 03:56:12 UTC on ccycloud.quasar-guijbm.root.comops.site
build id 4569618
ASAN enabled
I20260806 05:02:15.569180 3578 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 05:02:15.570422 3578 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 05:02:15.583493 3587 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 05:02:15.583474 3591 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 05:02:15.583458 3593 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 05:02:15.584645 3578 server_base.cc:1034] running on GCE node
I20260806 05:02:15.585198 3578 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 05:02:15.586148 3578 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 05:02:15.587421 3578 hybrid_clock.cc:648] HybridClock initialized: now 1785992535587349 us; error 42 us; skew 500 ppm
I20260806 05:02:15.587733 3578 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 05:02:15 dist-test-slave-x4qh krb5kdc[3029](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992535, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.96.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 05:02:15.590875 3578 init.cc:377] Logged in from keytab as kudu/127.9.96.67@KRBTEST.COM (short username kudu)
I20260806 05:02:15.592283 3578 webserver.cc:460] Webserver started at http://127.9.96.67:45147/ using document root <none> and password file <none>
I20260806 05:02:15.592942 3578 fs_manager.cc:359] Metadata directory not provided
I20260806 05:02:15.593087 3578 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 05:02:15.593394 3578 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 05:02:15.596247 3578 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785992227050689-9601-0/minicluster-data/ts-2/data/instance:
uuid: "cce539c2ff6c4af1ab6c74e523fbc3cc"
format_stamp: "Formatted at 2026-08-06 05:02:15 on dist-test-slave-x4qh"
server_key: "d5ee44092c5a46ad957a2b2c00f33a23"
server_key_iv: "3ea8d4312c99cb7b79aeab3d8fa949a5"
server_key_version: "encryptionkey@0"
I20260806 05:02:15.597047 3578 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785992227050689-9601-0/minicluster-data/ts-2/wal/instance:
uuid: "cce539c2ff6c4af1ab6c74e523fbc3cc"
format_stamp: "Formatted at 2026-08-06 05:02:15 on dist-test-slave-x4qh"
server_key: "d5ee44092c5a46ad957a2b2c00f33a23"
server_key_iv: "3ea8d4312c99cb7b79aeab3d8fa949a5"
server_key_version: "encryptionkey@0"
I20260806 05:02:15.603809 3578 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260806 05:02:15.607210 3610 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 05:02:15.608878 3578 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 05:02:15.609047 3578 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785992227050689-9601-0/minicluster-data/ts-2/data,/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785992227050689-9601-0/minicluster-data/ts-2/wal
uuid: "cce539c2ff6c4af1ab6c74e523fbc3cc"
format_stamp: "Formatted at 2026-08-06 05:02:15 on dist-test-slave-x4qh"
server_key: "d5ee44092c5a46ad957a2b2c00f33a23"
server_key_iv: "3ea8d4312c99cb7b79aeab3d8fa949a5"
server_key_version: "encryptionkey@0"
I20260806 05:02:15.609242 3578 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785992227050689-9601-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785992227050689-9601-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785992227050689-9601-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 05:02:15.621780 3578 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 05:02:15.625964 3578 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 05:02:15.626259 3578 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 05:02:15.627161 3578 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 05:02:15.628539 3578 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 05:02:15.628647 3578 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 05:02:15.628799 3578 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 05:02:15.628885 3578 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 05:02:15.633014 3368 heartbeater.cc:503] Master 127.9.96.126:46585 was elected leader, sending a full tablet report...
I20260806 05:02:15.663740 3578 rpc_server.cc:307] RPC server started. Bound to: 127.9.96.67:36307
I20260806 05:02:15.663864 3779 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.96.67:36307 every 8 connection(s)
I20260806 05:02:15.665058 3578 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785992227050689-9601-0/minicluster-data/ts-2/data/info.pb
I20260806 05:02:15.666898 9601 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu as pid 3578
I20260806 05:02:15.667076 9601 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785992227050689-9601-0/minicluster-data/ts-2/wal/instance
I20260806 05:02:15.667641 9601 external_mini_cluster.cc:1428] Setting key ffc46e2306706c87bf5001062ad91009
Aug 06 05:02:15 dist-test-slave-x4qh krb5kdc[3029](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992535, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.96.67@KRBTEST.COM for kudu/127.9.96.126@KRBTEST.COM
I20260806 05:02:15.686020 3785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:02:15.667869 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.96.67:37997 (local address 127.9.96.126:46585)
0806 05:02:15.668417 (+ 548us) server_negotiation.cc:206] Beginning negotiation
0806 05:02:15.668424 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 05:02:15.669618 (+ 1194us) server_negotiation.cc:407] Connection header received
0806 05:02:15.671948 (+ 2330us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 05:02:15.671953 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 05:02:15.672045 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 05:02:15.672156 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0806 05:02:15.673578 (+ 1422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:02:15.674407 (+ 829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:02:15.676223 (+ 1816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:02:15.676648 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:02:15.676756 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:02:15.680091 (+ 3335us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 05:02:15.680095 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 05:02:15.680099 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 05:02:15.680144 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0806 05:02:15.682907 (+ 2763us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 05:02:15.683701 (+ 794us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 05:02:15.683706 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 05:02:15.683712 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 05:02:15.683837 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 05:02:15.684279 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 05:02:15.684283 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 05:02:15.684288 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 05:02:15.684582 (+ 294us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 05:02:15.684782 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0806 05:02:15.685428 (+ 646us) server_negotiation.cc:299] Negotiation successful
0806 05:02:15.685643 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":420,"thread_start_us":288,"threads_started":1}
I20260806 05:02:15.686913 3786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:02:15.668411 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:46585 (local address 127.9.96.67:37997)
0806 05:02:15.669405 (+ 994us) negotiation.cc:107] Waiting for socket to connect
0806 05:02:15.669466 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0806 05:02:15.671602 (+ 2136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 05:02:15.672353 (+ 751us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 05:02:15.672360 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 05:02:15.672857 (+ 497us) client_negotiation.cc:189] Negotiated authn=SASL
0806 05:02:15.673415 (+ 558us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 05:02:15.673431 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 05:02:15.674587 (+ 1156us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 05:02:15.674591 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 05:02:15.676088 (+ 1497us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 05:02:15.676100 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 05:02:15.676849 (+ 749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 05:02:15.676854 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 05:02:15.677040 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:02:15.677735 (+ 695us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 05:02:15.677764 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0806 05:02:15.679902 (+ 2138us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 05:02:15.683060 (+ 3158us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 05:02:15.683069 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 05:02:15.683083 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0806 05:02:15.683573 (+ 490us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 05:02:15.683951 (+ 378us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 05:02:15.683956 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 05:02:15.683960 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 05:02:15.684144 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 05:02:15.684725 (+ 581us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 05:02:15.684733 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 05:02:15.685044 (+ 311us) client_negotiation.cc:769] Sending connection context
0806 05:02:15.685299 (+ 255us) client_negotiation.cc:240] Negotiation successful
0806 05:02:15.685789 (+ 490us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":626,"thread_start_us":359,"threads_started":1}
I20260806 05:02:15.688158 3781 heartbeater.cc:347] Connected to a master server at 127.9.96.126:46585
I20260806 05:02:15.688463 3781 heartbeater.cc:464] Registering TS with master...
I20260806 05:02:15.689157 3781 heartbeater.cc:511] Master 127.9.96.126:46585 requested a full tablet report, sending...
I20260806 05:02:15.690595 3104 ts_manager.cc:194] Registered new tserver with Master: cce539c2ff6c4af1ab6c74e523fbc3cc (127.9.96.67:36307)
I20260806 05:02:15.691601 3104 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.96.67@KRBTEST.COM'} at 127.9.96.67:37997
I20260806 05:02:15.695596 9601 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 05:02:15.713958 3785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:02:15.700029 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58574 (local address 127.9.96.126:46585)
0806 05:02:15.700181 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0806 05:02:15.700185 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 05:02:15.700642 (+ 457us) server_negotiation.cc:407] Connection header received
0806 05:02:15.700762 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 05:02:15.700765 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 05:02:15.700827 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 05:02:15.700913 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0806 05:02:15.702352 (+ 1439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:02:15.703200 (+ 848us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:02:15.704585 (+ 1385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:02:15.704989 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:02:15.705097 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:02:15.707800 (+ 2703us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 05:02:15.707805 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 05:02:15.707808 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 05:02:15.707849 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0806 05:02:15.710822 (+ 2973us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 05:02:15.711677 (+ 855us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 05:02:15.711686 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 05:02:15.711690 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 05:02:15.711844 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 05:02:15.712195 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 05:02:15.712197 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 05:02:15.712199 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 05:02:15.712643 (+ 444us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 05:02:15.712948 (+ 305us) server_negotiation.cc:1035] Waiting for connection context
0806 05:02:15.713343 (+ 395us) server_negotiation.cc:299] Negotiation successful
0806 05:02:15.713574 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260806 05:02:15.717792 3104 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58574:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 05:02:15.725037 3104 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 05:02:15.748891 3806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:02:15.737449 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.65:39517 (local address 127.0.0.1:36956)
0806 05:02:15.738689 (+ 1240us) negotiation.cc:107] Waiting for socket to connect
0806 05:02:15.738711 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0806 05:02:15.738862 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 05:02:15.739781 (+ 919us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 05:02:15.739788 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 05:02:15.739822 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 05:02:15.740297 (+ 475us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 05:02:15.740316 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 05:02:15.742667 (+ 2351us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 05:02:15.742673 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 05:02:15.745826 (+ 3153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 05:02:15.745842 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 05:02:15.748099 (+ 2257us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 05:02:15.748104 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 05:02:15.748282 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:02:15.748312 (+ 30us) client_negotiation.cc:769] Sending connection context
0806 05:02:15.748391 (+ 79us) client_negotiation.cc:240] Negotiation successful
0806 05:02:15.748494 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1091,"mutex_wait_us":298,"thread_start_us":943,"threads_started":1}
I20260806 05:02:15.749841 3804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:02:15.737382 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59230 (local address 127.9.96.66:42971)
0806 05:02:15.739426 (+ 2044us) server_negotiation.cc:206] Beginning negotiation
0806 05:02:15.739436 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 05:02:15.739463 (+ 27us) server_negotiation.cc:407] Connection header received
0806 05:02:15.739557 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 05:02:15.739568 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 05:02:15.739789 (+ 221us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 05:02:15.740693 (+ 904us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 05:02:15.740922 (+ 229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:02:15.742503 (+ 1581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:02:15.746015 (+ 3512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:02:15.747559 (+ 1544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:02:15.749024 (+ 1465us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:02:15.749210 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0806 05:02:15.749309 (+ 99us) server_negotiation.cc:299] Negotiation successful
0806 05:02:15.749480 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1844,"thread_start_us":2586,"threads_started":1}
I20260806 05:02:15.748924 3799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:02:15.737046 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.66:42971 (local address 127.0.0.1:59230)
0806 05:02:15.738063 (+ 1017us) negotiation.cc:107] Waiting for socket to connect
0806 05:02:15.738106 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0806 05:02:15.738278 (+ 172us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 05:02:15.740022 (+ 1744us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 05:02:15.740032 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 05:02:15.740052 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 05:02:15.740512 (+ 460us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 05:02:15.740523 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 05:02:15.742667 (+ 2144us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 05:02:15.742673 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 05:02:15.745826 (+ 3153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 05:02:15.745842 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 05:02:15.747794 (+ 1952us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 05:02:15.747801 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 05:02:15.748023 (+ 222us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:02:15.748097 (+ 74us) client_negotiation.cc:769] Sending connection context
0806 05:02:15.748207 (+ 110us) client_negotiation.cc:240] Negotiation successful
0806 05:02:15.748407 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":788,"mutex_wait_us":191,"thread_start_us":637,"threads_started":1}
I20260806 05:02:15.750471 3807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:02:15.738027 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36956 (local address 127.9.96.65:39517)
0806 05:02:15.739055 (+ 1028us) server_negotiation.cc:206] Beginning negotiation
0806 05:02:15.739066 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 05:02:15.739094 (+ 28us) server_negotiation.cc:407] Connection header received
0806 05:02:15.739198 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 05:02:15.739205 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 05:02:15.739469 (+ 264us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 05:02:15.740558 (+ 1089us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 05:02:15.740846 (+ 288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:02:15.742503 (+ 1657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:02:15.746015 (+ 3512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:02:15.747559 (+ 1544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:02:15.747696 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:02:15.747952 (+ 256us) server_negotiation.cc:1035] Waiting for connection context
0806 05:02:15.749964 (+ 2012us) server_negotiation.cc:299] Negotiation successful
0806 05:02:15.750133 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":816,"thread_start_us":622,"threads_started":1}
I20260806 05:02:15.754462 3280 tablet_service.cc:1467] Processing CreateTablet for tablet 3b686a92fe194cd6b2e3d91024d257f3 (DEFAULT_TABLE table=test-table [id=f72282d6d50e437b9cbf15986bcf6c33]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 05:02:15.754524 3472 tablet_service.cc:1467] Processing CreateTablet for tablet 3b686a92fe194cd6b2e3d91024d257f3 (DEFAULT_TABLE table=test-table [id=f72282d6d50e437b9cbf15986bcf6c33]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 05:02:15.755919 3280 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3b686a92fe194cd6b2e3d91024d257f3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 05:02:15.755972 3472 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3b686a92fe194cd6b2e3d91024d257f3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 05:02:15.759117 3806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:02:15.739390 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.67:36307 (local address 127.0.0.1:50524)
0806 05:02:15.749414 (+ 10024us) negotiation.cc:107] Waiting for socket to connect
0806 05:02:15.749432 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 05:02:15.749523 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 05:02:15.751802 (+ 2279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 05:02:15.751807 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 05:02:15.751837 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 05:02:15.752032 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 05:02:15.752046 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 05:02:15.754022 (+ 1976us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 05:02:15.754028 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 05:02:15.756460 (+ 2432us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 05:02:15.756477 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 05:02:15.758375 (+ 1898us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 05:02:15.758381 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 05:02:15.758572 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:02:15.758608 (+ 36us) client_negotiation.cc:769] Sending connection context
0806 05:02:15.758702 (+ 94us) client_negotiation.cc:240] Negotiation successful
0806 05:02:15.758854 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9937,"thread_start_us":11717,"threads_started":1}
I20260806 05:02:15.759341 3810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:02:15.740021 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50524 (local address 127.9.96.67:36307)
0806 05:02:15.751203 (+ 11182us) server_negotiation.cc:206] Beginning negotiation
0806 05:02:15.751211 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 05:02:15.751240 (+ 29us) server_negotiation.cc:407] Connection header received
0806 05:02:15.751343 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 05:02:15.751350 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 05:02:15.751588 (+ 238us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 05:02:15.751798 (+ 210us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 05:02:15.752220 (+ 422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:02:15.753854 (+ 1634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:02:15.756674 (+ 2820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:02:15.758192 (+ 1518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:02:15.758348 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:02:15.758571 (+ 223us) server_negotiation.cc:1035] Waiting for connection context
0806 05:02:15.758772 (+ 201us) server_negotiation.cc:299] Negotiation successful
0806 05:02:15.758949 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11022,"thread_start_us":10874,"threads_started":1}
I20260806 05:02:15.763101 3681 tablet_service.cc:1467] Processing CreateTablet for tablet 3b686a92fe194cd6b2e3d91024d257f3 (DEFAULT_TABLE table=test-table [id=f72282d6d50e437b9cbf15986bcf6c33]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 05:02:15.764603 3681 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3b686a92fe194cd6b2e3d91024d257f3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 05:02:15.767722 3812 tablet_bootstrap.cc:492] T 3b686a92fe194cd6b2e3d91024d257f3 P 62bb147239a74aab8c7fe7f7f90a757e: Bootstrap starting.
I20260806 05:02:15.771080 3812 tablet_bootstrap.cc:654] T 3b686a92fe194cd6b2e3d91024d257f3 P 62bb147239a74aab8c7fe7f7f90a757e: Neither blocks nor log segments found. Creating new log.
I20260806 05:02:15.772123 3813 tablet_bootstrap.cc:492] T 3b686a92fe194cd6b2e3d91024d257f3 P 6b2713f209924713a4e746ced443630e: Bootstrap starting.
I20260806 05:02:15.772553 3812 log.cc:824] T 3b686a92fe194cd6b2e3d91024d257f3 P 62bb147239a74aab8c7fe7f7f90a757e: Log is configured to *not* fsync() on all Append() calls
I20260806 05:02:15.775736 3813 tablet_bootstrap.cc:654] T 3b686a92fe194cd6b2e3d91024d257f3 P 6b2713f209924713a4e746ced443630e: Neither blocks nor log segments found. Creating new log.
I20260806 05:02:15.776067 3812 tablet_bootstrap.cc:492] T 3b686a92fe194cd6b2e3d91024d257f3 P 62bb147239a74aab8c7fe7f7f90a757e: No bootstrap required, opened a new log
I20260806 05:02:15.776356 3812 ts_tablet_manager.cc:1397] T 3b686a92fe194cd6b2e3d91024d257f3 P 62bb147239a74aab8c7fe7f7f90a757e: Time spent bootstrapping tablet: real 0.009s user 0.004s sys 0.003s
I20260806 05:02:15.777012 3813 log.cc:824] T 3b686a92fe194cd6b2e3d91024d257f3 P 6b2713f209924713a4e746ced443630e: Log is configured to *not* fsync() on all Append() calls
I20260806 05:02:15.780230 3813 tablet_bootstrap.cc:492] T 3b686a92fe194cd6b2e3d91024d257f3 P 6b2713f209924713a4e746ced443630e: No bootstrap required, opened a new log
I20260806 05:02:15.780586 3813 ts_tablet_manager.cc:1397] T 3b686a92fe194cd6b2e3d91024d257f3 P 6b2713f209924713a4e746ced443630e: Time spent bootstrapping tablet: real 0.009s user 0.003s sys 0.005s
I20260806 05:02:15.780843 3816 tablet_bootstrap.cc:492] T 3b686a92fe194cd6b2e3d91024d257f3 P cce539c2ff6c4af1ab6c74e523fbc3cc: Bootstrap starting.
I20260806 05:02:15.784668 3816 tablet_bootstrap.cc:654] T 3b686a92fe194cd6b2e3d91024d257f3 P cce539c2ff6c4af1ab6c74e523fbc3cc: Neither blocks nor log segments found. Creating new log.
I20260806 05:02:15.785233 3812 raft_consensus.cc:348] T 3b686a92fe194cd6b2e3d91024d257f3 P 62bb147239a74aab8c7fe7f7f90a757e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b2713f209924713a4e746ced443630e" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 42971 } } peers { permanent_uuid: "62bb147239a74aab8c7fe7f7f90a757e" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 39517 } } peers { permanent_uuid: "cce539c2ff6c4af1ab6c74e523fbc3cc" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 36307 } }
I20260806 05:02:15.785596 3812 raft_consensus.cc:374] T 3b686a92fe194cd6b2e3d91024d257f3 P 62bb147239a74aab8c7fe7f7f90a757e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 05:02:15.785727 3812 raft_consensus.cc:729] T 3b686a92fe194cd6b2e3d91024d257f3 P 62bb147239a74aab8c7fe7f7f90a757e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 62bb147239a74aab8c7fe7f7f90a757e, State: Initialized, Role: FOLLOWER
I20260806 05:02:15.785971 3816 log.cc:824] T 3b686a92fe194cd6b2e3d91024d257f3 P cce539c2ff6c4af1ab6c74e523fbc3cc: Log is configured to *not* fsync() on all Append() calls
I20260806 05:02:15.786317 3812 consensus_queue.cc:260] T 3b686a92fe194cd6b2e3d91024d257f3 P 62bb147239a74aab8c7fe7f7f90a757e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b2713f209924713a4e746ced443630e" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 42971 } } peers { permanent_uuid: "62bb147239a74aab8c7fe7f7f90a757e" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 39517 } } peers { permanent_uuid: "cce539c2ff6c4af1ab6c74e523fbc3cc" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 36307 } }
I20260806 05:02:15.788028 3812 ts_tablet_manager.cc:1428] T 3b686a92fe194cd6b2e3d91024d257f3 P 62bb147239a74aab8c7fe7f7f90a757e: Time spent starting tablet: real 0.012s user 0.007s sys 0.005s
I20260806 05:02:15.789253 3816 tablet_bootstrap.cc:492] T 3b686a92fe194cd6b2e3d91024d257f3 P cce539c2ff6c4af1ab6c74e523fbc3cc: No bootstrap required, opened a new log
I20260806 05:02:15.789582 3816 ts_tablet_manager.cc:1397] T 3b686a92fe194cd6b2e3d91024d257f3 P cce539c2ff6c4af1ab6c74e523fbc3cc: Time spent bootstrapping tablet: real 0.009s user 0.000s sys 0.006s
I20260806 05:02:15.792114 3813 raft_consensus.cc:348] T 3b686a92fe194cd6b2e3d91024d257f3 P 6b2713f209924713a4e746ced443630e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b2713f209924713a4e746ced443630e" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 42971 } } peers { permanent_uuid: "62bb147239a74aab8c7fe7f7f90a757e" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 39517 } } peers { permanent_uuid: "cce539c2ff6c4af1ab6c74e523fbc3cc" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 36307 } }
I20260806 05:02:15.792495 3813 raft_consensus.cc:374] T 3b686a92fe194cd6b2e3d91024d257f3 P 6b2713f209924713a4e746ced443630e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 05:02:15.792608 3813 raft_consensus.cc:729] T 3b686a92fe194cd6b2e3d91024d257f3 P 6b2713f209924713a4e746ced443630e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6b2713f209924713a4e746ced443630e, State: Initialized, Role: FOLLOWER
I20260806 05:02:15.793195 3813 consensus_queue.cc:260] T 3b686a92fe194cd6b2e3d91024d257f3 P 6b2713f209924713a4e746ced443630e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b2713f209924713a4e746ced443630e" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 42971 } } peers { permanent_uuid: "62bb147239a74aab8c7fe7f7f90a757e" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 39517 } } peers { permanent_uuid: "cce539c2ff6c4af1ab6c74e523fbc3cc" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 36307 } }
I20260806 05:02:15.794265 3566 heartbeater.cc:503] Master 127.9.96.126:46585 was elected leader, sending a full tablet report...
I20260806 05:02:15.794847 3813 ts_tablet_manager.cc:1428] T 3b686a92fe194cd6b2e3d91024d257f3 P 6b2713f209924713a4e746ced443630e: Time spent starting tablet: real 0.014s user 0.012s sys 0.000s
I20260806 05:02:15.799809 3816 raft_consensus.cc:348] T 3b686a92fe194cd6b2e3d91024d257f3 P cce539c2ff6c4af1ab6c74e523fbc3cc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b2713f209924713a4e746ced443630e" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 42971 } } peers { permanent_uuid: "62bb147239a74aab8c7fe7f7f90a757e" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 39517 } } peers { permanent_uuid: "cce539c2ff6c4af1ab6c74e523fbc3cc" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 36307 } }
I20260806 05:02:15.800177 3816 raft_consensus.cc:374] T 3b686a92fe194cd6b2e3d91024d257f3 P cce539c2ff6c4af1ab6c74e523fbc3cc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 05:02:15.800300 3816 raft_consensus.cc:729] T 3b686a92fe194cd6b2e3d91024d257f3 P cce539c2ff6c4af1ab6c74e523fbc3cc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cce539c2ff6c4af1ab6c74e523fbc3cc, State: Initialized, Role: FOLLOWER
I20260806 05:02:15.800853 3816 consensus_queue.cc:260] T 3b686a92fe194cd6b2e3d91024d257f3 P cce539c2ff6c4af1ab6c74e523fbc3cc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b2713f209924713a4e746ced443630e" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 42971 } } peers { permanent_uuid: "62bb147239a74aab8c7fe7f7f90a757e" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 39517 } } peers { permanent_uuid: "cce539c2ff6c4af1ab6c74e523fbc3cc" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 36307 } }
I20260806 05:02:15.801892 3781 heartbeater.cc:503] Master 127.9.96.126:46585 was elected leader, sending a full tablet report...
I20260806 05:02:15.802310 3816 ts_tablet_manager.cc:1428] T 3b686a92fe194cd6b2e3d91024d257f3 P cce539c2ff6c4af1ab6c74e523fbc3cc: Time spent starting tablet: real 0.013s user 0.012s sys 0.000s
W20260806 05:02:15.852458 3370 tablet.cc:2369] T 3b686a92fe194cd6b2e3d91024d257f3 P 62bb147239a74aab8c7fe7f7f90a757e: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 05:02:15.885234 3572 tablet.cc:2369] T 3b686a92fe194cd6b2e3d91024d257f3 P 6b2713f209924713a4e746ced443630e: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 05:02:15.916604 3783 tablet.cc:2369] T 3b686a92fe194cd6b2e3d91024d257f3 P cce539c2ff6c4af1ab6c74e523fbc3cc: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 05:02:16.085035 3824 raft_consensus.cc:482] T 3b686a92fe194cd6b2e3d91024d257f3 P cce539c2ff6c4af1ab6c74e523fbc3cc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 05:02:16.085361 3824 raft_consensus.cc:504] T 3b686a92fe194cd6b2e3d91024d257f3 P cce539c2ff6c4af1ab6c74e523fbc3cc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b2713f209924713a4e746ced443630e" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 42971 } } peers { permanent_uuid: "62bb147239a74aab8c7fe7f7f90a757e" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 39517 } } peers { permanent_uuid: "cce539c2ff6c4af1ab6c74e523fbc3cc" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 36307 } }
I20260806 05:02:16.086316 3820 raft_consensus.cc:482] T 3b686a92fe194cd6b2e3d91024d257f3 P 62bb147239a74aab8c7fe7f7f90a757e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 05:02:16.086593 3820 raft_consensus.cc:504] T 3b686a92fe194cd6b2e3d91024d257f3 P 62bb147239a74aab8c7fe7f7f90a757e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b2713f209924713a4e746ced443630e" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 42971 } } peers { permanent_uuid: "62bb147239a74aab8c7fe7f7f90a757e" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 39517 } } peers { permanent_uuid: "cce539c2ff6c4af1ab6c74e523fbc3cc" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 36307 } }
I20260806 05:02:16.087217 3824 leader_election.cc:288] T 3b686a92fe194cd6b2e3d91024d257f3 P cce539c2ff6c4af1ab6c74e523fbc3cc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6b2713f209924713a4e746ced443630e (127.9.96.66:42971), 62bb147239a74aab8c7fe7f7f90a757e (127.9.96.65:39517)
I20260806 05:02:16.088656 3820 leader_election.cc:288] T 3b686a92fe194cd6b2e3d91024d257f3 P 62bb147239a74aab8c7fe7f7f90a757e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6b2713f209924713a4e746ced443630e (127.9.96.66:42971), cce539c2ff6c4af1ab6c74e523fbc3cc (127.9.96.67:36307)
I20260806 05:02:16.097841 3786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:02:16.087635 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.66:42971 (local address 127.9.96.67:59351)
0806 05:02:16.087954 (+ 319us) negotiation.cc:107] Waiting for socket to connect
0806 05:02:16.087975 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0806 05:02:16.088137 (+ 162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 05:02:16.088795 (+ 658us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 05:02:16.088803 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 05:02:16.088843 (+ 40us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 05:02:16.091307 (+ 2464us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 05:02:16.091324 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 05:02:16.092981 (+ 1657us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 05:02:16.092985 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 05:02:16.095121 (+ 2136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 05:02:16.095142 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 05:02:16.097008 (+ 1866us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 05:02:16.097015 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 05:02:16.097223 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:02:16.097275 (+ 52us) client_negotiation.cc:769] Sending connection context
0806 05:02:16.097387 (+ 112us) client_negotiation.cc:240] Negotiation successful
0806 05:02:16.097532 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":187}
I20260806 05:02:16.098610 3804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:02:16.087726 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.96.67:59351 (local address 127.9.96.66:42971)
0806 05:02:16.087982 (+ 256us) server_negotiation.cc:206] Beginning negotiation
0806 05:02:16.087988 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 05:02:16.088074 (+ 86us) server_negotiation.cc:407] Connection header received
0806 05:02:16.088341 (+ 267us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 05:02:16.088350 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 05:02:16.088438 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 05:02:16.088547 (+ 109us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 05:02:16.091562 (+ 3015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:02:16.092745 (+ 1183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:02:16.095326 (+ 2581us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:02:16.096622 (+ 1296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:02:16.096788 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:02:16.096905 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0806 05:02:16.098184 (+ 1279us) server_negotiation.cc:299] Negotiation successful
0806 05:02:16.098334 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":6,"server-negotiator.queue_time_us":106}
I20260806 05:02:16.098991 3828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:02:16.088969 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.66:42971 (local address 127.9.96.65:45337)
0806 05:02:16.089806 (+ 837us) negotiation.cc:107] Waiting for socket to connect
0806 05:02:16.089830 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0806 05:02:16.089984 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 05:02:16.091799 (+ 1815us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 05:02:16.091804 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 05:02:16.091840 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 05:02:16.092268 (+ 428us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 05:02:16.092280 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 05:02:16.093962 (+ 1682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 05:02:16.093970 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 05:02:16.096243 (+ 2273us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 05:02:16.096260 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 05:02:16.098164 (+ 1904us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 05:02:16.098170 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 05:02:16.098362 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:02:16.098397 (+ 35us) client_negotiation.cc:769] Sending connection context
0806 05:02:16.098465 (+ 68us) client_negotiation.cc:240] Negotiation successful
0806 05:02:16.098593 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":651,"mutex_wait_us":19,"thread_start_us":403,"threads_started":1}
I20260806 05:02:16.099658 3829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:02:16.089269 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.96.65:45337 (local address 127.9.96.66:42971)
0806 05:02:16.091467 (+ 2198us) server_negotiation.cc:206] Beginning negotiation
0806 05:02:16.091473 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 05:02:16.091495 (+ 22us) server_negotiation.cc:407] Connection header received
0806 05:02:16.091560 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 05:02:16.091564 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 05:02:16.091632 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 05:02:16.092038 (+ 406us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 05:02:16.092632 (+ 594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:02:16.093827 (+ 1195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:02:16.096463 (+ 2636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:02:16.097935 (+ 1472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:02:16.098969 (+ 1034us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:02:16.099116 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0806 05:02:16.099166 (+ 50us) server_negotiation.cc:299] Negotiation successful
0806 05:02:16.099299 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":56,"server-negotiator.queue_time_us":2016,"thread_start_us":1766,"threads_started":1}
I20260806 05:02:16.099556 3501 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3b686a92fe194cd6b2e3d91024d257f3" candidate_uuid: "cce539c2ff6c4af1ab6c74e523fbc3cc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b2713f209924713a4e746ced443630e" is_pre_election: true
I20260806 05:02:16.100133 3810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:02:16.091313 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.96.65:53723 (local address 127.9.96.67:36307)
0806 05:02:16.091509 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0806 05:02:16.091515 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 05:02:16.091533 (+ 18us) server_negotiation.cc:407] Connection header received
0806 05:02:16.091588 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 05:02:16.091592 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 05:02:16.091649 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 05:02:16.091745 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 05:02:16.093038 (+ 1293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:02:16.095010 (+ 1972us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:02:16.097636 (+ 2626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:02:16.098928 (+ 1292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:02:16.099035 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:02:16.099133 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0806 05:02:16.099714 (+ 581us) server_negotiation.cc:299] Negotiation successful
0806 05:02:16.099847 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":672,"server-negotiator.queue_time_us":37}
I20260806 05:02:16.100224 3501 raft_consensus.cc:2454] T 3b686a92fe194cd6b2e3d91024d257f3 P 6b2713f209924713a4e746ced443630e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cce539c2ff6c4af1ab6c74e523fbc3cc in term 0.
I20260806 05:02:16.100412 3807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:02:16.087799 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.96.67:42243 (local address 127.9.96.65:39517)
0806 05:02:16.087984 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0806 05:02:16.087990 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 05:02:16.091840 (+ 3850us) server_negotiation.cc:407] Connection header received
0806 05:02:16.092092 (+ 252us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 05:02:16.092097 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 05:02:16.092169 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 05:02:16.092251 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 05:02:16.093140 (+ 889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:02:16.094295 (+ 1155us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:02:16.096831 (+ 2536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:02:16.098173 (+ 1342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:02:16.098287 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:02:16.098373 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0806 05:02:16.100035 (+ 1662us) server_negotiation.cc:299] Negotiation successful
0806 05:02:16.100170 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":60}
I20260806 05:02:16.100632 3500 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3b686a92fe194cd6b2e3d91024d257f3" candidate_uuid: "62bb147239a74aab8c7fe7f7f90a757e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b2713f209924713a4e746ced443630e" is_pre_election: true
I20260806 05:02:16.100961 3830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:02:16.089938 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.67:36307 (local address 127.9.96.65:53723)
0806 05:02:16.090636 (+ 698us) negotiation.cc:107] Waiting for socket to connect
0806 05:02:16.090658 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0806 05:02:16.090793 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 05:02:16.091889 (+ 1096us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 05:02:16.091898 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 05:02:16.091996 (+ 98us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 05:02:16.092798 (+ 802us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 05:02:16.092814 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 05:02:16.095183 (+ 2369us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 05:02:16.095190 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 05:02:16.097425 (+ 2235us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 05:02:16.097441 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 05:02:16.099355 (+ 1914us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 05:02:16.099362 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 05:02:16.099530 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:02:16.099558 (+ 28us) client_negotiation.cc:769] Sending connection context
0806 05:02:16.100279 (+ 721us) client_negotiation.cc:240] Negotiation successful
0806 05:02:16.100404 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":548,"mutex_wait_us":35,"spinlock_wait_cycles":23936,"thread_start_us":379,"threads_started":1}
I20260806 05:02:16.100984 3500 raft_consensus.cc:2454] T 3b686a92fe194cd6b2e3d91024d257f3 P 6b2713f209924713a4e746ced443630e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 62bb147239a74aab8c7fe7f7f90a757e in term 0.
I20260806 05:02:16.101243 3611 leader_election.cc:302] T 3b686a92fe194cd6b2e3d91024d257f3 P cce539c2ff6c4af1ab6c74e523fbc3cc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b2713f209924713a4e746ced443630e, cce539c2ff6c4af1ab6c74e523fbc3cc; no voters:
I20260806 05:02:16.101646 3827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:02:16.088295 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.65:39517 (local address 127.9.96.67:42243)
0806 05:02:16.091747 (+ 3452us) negotiation.cc:107] Waiting for socket to connect
0806 05:02:16.091764 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 05:02:16.091940 (+ 176us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 05:02:16.092442 (+ 502us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 05:02:16.092446 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 05:02:16.092469 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 05:02:16.092883 (+ 414us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 05:02:16.092897 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 05:02:16.094487 (+ 1590us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 05:02:16.094492 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 05:02:16.096644 (+ 2152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 05:02:16.096660 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 05:02:16.098623 (+ 1963us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 05:02:16.098632 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 05:02:16.098836 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:02:16.099700 (+ 864us) client_negotiation.cc:769] Sending connection context
0806 05:02:16.100714 (+ 1014us) client_negotiation.cc:240] Negotiation successful
0806 05:02:16.100848 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3316,"mutex_wait_us":787,"thread_start_us":4003,"threads_started":1}
I20260806 05:02:16.101701 3824 raft_consensus.cc:2781] T 3b686a92fe194cd6b2e3d91024d257f3 P cce539c2ff6c4af1ab6c74e523fbc3cc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 05:02:16.101629 3211 leader_election.cc:302] T 3b686a92fe194cd6b2e3d91024d257f3 P 62bb147239a74aab8c7fe7f7f90a757e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 62bb147239a74aab8c7fe7f7f90a757e, 6b2713f209924713a4e746ced443630e; no voters:
I20260806 05:02:16.101867 3824 raft_consensus.cc:482] T 3b686a92fe194cd6b2e3d91024d257f3 P cce539c2ff6c4af1ab6c74e523fbc3cc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 05:02:16.101976 3824 raft_consensus.cc:3037] T 3b686a92fe194cd6b2e3d91024d257f3 P cce539c2ff6c4af1ab6c74e523fbc3cc [term 0 FOLLOWER]: Advancing to term 1
I20260806 05:02:16.102126 3820 raft_consensus.cc:2781] T 3b686a92fe194cd6b2e3d91024d257f3 P 62bb147239a74aab8c7fe7f7f90a757e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 05:02:16.102293 3820 raft_consensus.cc:482] T 3b686a92fe194cd6b2e3d91024d257f3 P 62bb147239a74aab8c7fe7f7f90a757e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 05:02:16.102418 3820 raft_consensus.cc:3037] T 3b686a92fe194cd6b2e3d91024d257f3 P 62bb147239a74aab8c7fe7f7f90a757e [term 0 FOLLOWER]: Advancing to term 1
I20260806 05:02:16.102672 3308 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3b686a92fe194cd6b2e3d91024d257f3" candidate_uuid: "cce539c2ff6c4af1ab6c74e523fbc3cc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "62bb147239a74aab8c7fe7f7f90a757e" is_pre_election: true
I20260806 05:02:16.102694 3712 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3b686a92fe194cd6b2e3d91024d257f3" candidate_uuid: "62bb147239a74aab8c7fe7f7f90a757e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cce539c2ff6c4af1ab6c74e523fbc3cc" is_pre_election: true
I20260806 05:02:16.105140 3824 raft_consensus.cc:504] T 3b686a92fe194cd6b2e3d91024d257f3 P cce539c2ff6c4af1ab6c74e523fbc3cc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b2713f209924713a4e746ced443630e" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 42971 } } peers { permanent_uuid: "62bb147239a74aab8c7fe7f7f90a757e" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 39517 } } peers { permanent_uuid: "cce539c2ff6c4af1ab6c74e523fbc3cc" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 36307 } }
I20260806 05:02:16.105456 3820 raft_consensus.cc:504] T 3b686a92fe194cd6b2e3d91024d257f3 P 62bb147239a74aab8c7fe7f7f90a757e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b2713f209924713a4e746ced443630e" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 42971 } } peers { permanent_uuid: "62bb147239a74aab8c7fe7f7f90a757e" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 39517 } } peers { permanent_uuid: "cce539c2ff6c4af1ab6c74e523fbc3cc" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 36307 } }
I20260806 05:02:16.105798 3712 raft_consensus.cc:2379] T 3b686a92fe194cd6b2e3d91024d257f3 P cce539c2ff6c4af1ab6c74e523fbc3cc [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 62bb147239a74aab8c7fe7f7f90a757e in current term 1: Already voted for candidate cce539c2ff6c4af1ab6c74e523fbc3cc in this term.
I20260806 05:02:16.105894 3824 leader_election.cc:288] T 3b686a92fe194cd6b2e3d91024d257f3 P cce539c2ff6c4af1ab6c74e523fbc3cc [CANDIDATE]: Term 1 election: Requested vote from peers 6b2713f209924713a4e746ced443630e (127.9.96.66:42971), 62bb147239a74aab8c7fe7f7f90a757e (127.9.96.65:39517)
I20260806 05:02:16.105958 3308 raft_consensus.cc:2379] T 3b686a92fe194cd6b2e3d91024d257f3 P 62bb147239a74aab8c7fe7f7f90a757e [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cce539c2ff6c4af1ab6c74e523fbc3cc in current term 1: Already voted for candidate 62bb147239a74aab8c7fe7f7f90a757e in this term.
I20260806 05:02:16.106261 3820 leader_election.cc:288] T 3b686a92fe194cd6b2e3d91024d257f3 P 62bb147239a74aab8c7fe7f7f90a757e [CANDIDATE]: Term 1 election: Requested vote from peers 6b2713f209924713a4e746ced443630e (127.9.96.66:42971), cce539c2ff6c4af1ab6c74e523fbc3cc (127.9.96.67:36307)
I20260806 05:02:16.107537 3500 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3b686a92fe194cd6b2e3d91024d257f3" candidate_uuid: "cce539c2ff6c4af1ab6c74e523fbc3cc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b2713f209924713a4e746ced443630e"
I20260806 05:02:16.107558 3307 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3b686a92fe194cd6b2e3d91024d257f3" candidate_uuid: "cce539c2ff6c4af1ab6c74e523fbc3cc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "62bb147239a74aab8c7fe7f7f90a757e"
I20260806 05:02:16.107750 3500 raft_consensus.cc:3037] T 3b686a92fe194cd6b2e3d91024d257f3 P 6b2713f209924713a4e746ced443630e [term 0 FOLLOWER]: Advancing to term 1
I20260806 05:02:16.107957 3307 raft_consensus.cc:2379] T 3b686a92fe194cd6b2e3d91024d257f3 P 62bb147239a74aab8c7fe7f7f90a757e [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate cce539c2ff6c4af1ab6c74e523fbc3cc in current term 1: Already voted for candidate 62bb147239a74aab8c7fe7f7f90a757e in this term.
I20260806 05:02:16.108462 3501 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3b686a92fe194cd6b2e3d91024d257f3" candidate_uuid: "62bb147239a74aab8c7fe7f7f90a757e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b2713f209924713a4e746ced443630e"
I20260806 05:02:16.108592 3712 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3b686a92fe194cd6b2e3d91024d257f3" candidate_uuid: "62bb147239a74aab8c7fe7f7f90a757e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cce539c2ff6c4af1ab6c74e523fbc3cc"
I20260806 05:02:16.108898 3712 raft_consensus.cc:2379] T 3b686a92fe194cd6b2e3d91024d257f3 P cce539c2ff6c4af1ab6c74e523fbc3cc [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 62bb147239a74aab8c7fe7f7f90a757e in current term 1: Already voted for candidate cce539c2ff6c4af1ab6c74e523fbc3cc in this term.
I20260806 05:02:16.109385 3211 leader_election.cc:302] T 3b686a92fe194cd6b2e3d91024d257f3 P 62bb147239a74aab8c7fe7f7f90a757e [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 62bb147239a74aab8c7fe7f7f90a757e; no voters: 6b2713f209924713a4e746ced443630e, cce539c2ff6c4af1ab6c74e523fbc3cc
I20260806 05:02:16.109721 3820 raft_consensus.cc:2726] T 3b686a92fe194cd6b2e3d91024d257f3 P 62bb147239a74aab8c7fe7f7f90a757e [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260806 05:02:16.110711 3500 raft_consensus.cc:2454] T 3b686a92fe194cd6b2e3d91024d257f3 P 6b2713f209924713a4e746ced443630e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cce539c2ff6c4af1ab6c74e523fbc3cc in term 1.
I20260806 05:02:16.111171 3611 leader_election.cc:302] T 3b686a92fe194cd6b2e3d91024d257f3 P cce539c2ff6c4af1ab6c74e523fbc3cc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6b2713f209924713a4e746ced443630e, cce539c2ff6c4af1ab6c74e523fbc3cc; no voters: 62bb147239a74aab8c7fe7f7f90a757e
I20260806 05:02:16.111522 3824 raft_consensus.cc:2781] T 3b686a92fe194cd6b2e3d91024d257f3 P cce539c2ff6c4af1ab6c74e523fbc3cc [term 1 FOLLOWER]: Leader election won for term 1
I20260806 05:02:16.111945 3824 raft_consensus.cc:686] T 3b686a92fe194cd6b2e3d91024d257f3 P cce539c2ff6c4af1ab6c74e523fbc3cc [term 1 LEADER]: Becoming Leader. State: Replica: cce539c2ff6c4af1ab6c74e523fbc3cc, State: Running, Role: LEADER
I20260806 05:02:16.112457 3824 consensus_queue.cc:237] T 3b686a92fe194cd6b2e3d91024d257f3 P cce539c2ff6c4af1ab6c74e523fbc3cc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b2713f209924713a4e746ced443630e" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 42971 } } peers { permanent_uuid: "62bb147239a74aab8c7fe7f7f90a757e" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 39517 } } peers { permanent_uuid: "cce539c2ff6c4af1ab6c74e523fbc3cc" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 36307 } }
I20260806 05:02:16.116466 3104 catalog_manager.cc:5495] T 3b686a92fe194cd6b2e3d91024d257f3 P cce539c2ff6c4af1ab6c74e523fbc3cc reported cstate change: term changed from 0 to 1, leader changed from <none> to cce539c2ff6c4af1ab6c74e523fbc3cc (127.9.96.67). New cstate: current_term: 1 leader_uuid: "cce539c2ff6c4af1ab6c74e523fbc3cc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b2713f209924713a4e746ced443630e" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 42971 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "62bb147239a74aab8c7fe7f7f90a757e" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 39517 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cce539c2ff6c4af1ab6c74e523fbc3cc" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 36307 } health_report { overall_health: HEALTHY } } }
I20260806 05:02:16.207161 3810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:02:16.201228 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50538 (local address 127.9.96.67:36307)
0806 05:02:16.201420 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0806 05:02:16.201425 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 05:02:16.201595 (+ 170us) server_negotiation.cc:407] Connection header received
0806 05:02:16.201851 (+ 256us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 05:02:16.201855 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 05:02:16.202014 (+ 159us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 05:02:16.202199 (+ 185us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 05:02:16.202716 (+ 517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:02:16.203748 (+ 1032us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:02:16.205136 (+ 1388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:02:16.205546 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:02:16.205653 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:02:16.206148 (+ 495us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 05:02:16.206638 (+ 490us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 05:02:16.206753 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0806 05:02:16.206861 (+ 108us) server_negotiation.cc:299] Negotiation successful
0806 05:02:16.206928 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260806 05:02:16.215586 3307 raft_consensus.cc:1261] T 3b686a92fe194cd6b2e3d91024d257f3 P 62bb147239a74aab8c7fe7f7f90a757e [term 1 FOLLOWER]: Refusing update from remote peer cce539c2ff6c4af1ab6c74e523fbc3cc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 05:02:16.215610 3500 raft_consensus.cc:1261] T 3b686a92fe194cd6b2e3d91024d257f3 P 6b2713f209924713a4e746ced443630e [term 1 FOLLOWER]: Refusing update from remote peer cce539c2ff6c4af1ab6c74e523fbc3cc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 05:02:16.216523 3831 consensus_queue.cc:1035] T 3b686a92fe194cd6b2e3d91024d257f3 P cce539c2ff6c4af1ab6c74e523fbc3cc [LEADER]: Connected to new peer: Peer: permanent_uuid: "62bb147239a74aab8c7fe7f7f90a757e" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 39517 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 05:02:16.216921 3824 consensus_queue.cc:1035] T 3b686a92fe194cd6b2e3d91024d257f3 P cce539c2ff6c4af1ab6c74e523fbc3cc [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b2713f209924713a4e746ced443630e" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 42971 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 05:02:16.242381 3102 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58574:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 05:02:16.242676 3102 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58574:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 05:02:16.246088 3102 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 73e62ec4a04a4fd38a4e2724ac34dd1a: Sending DeleteTablet for 3 replicas of tablet 3b686a92fe194cd6b2e3d91024d257f3
I20260806 05:02:16.247138 3472 tablet_service.cc:1514] Processing DeleteTablet for tablet 3b686a92fe194cd6b2e3d91024d257f3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 05:02:16 UTC) from {username='kudu', principal='kudu/127.9.96.126@KRBTEST.COM'} at 127.0.0.1:59230
I20260806 05:02:16.247298 3280 tablet_service.cc:1514] Processing DeleteTablet for tablet 3b686a92fe194cd6b2e3d91024d257f3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 05:02:16 UTC) from {username='kudu', principal='kudu/127.9.96.126@KRBTEST.COM'} at 127.0.0.1:36956
I20260806 05:02:16.247538 3681 tablet_service.cc:1514] Processing DeleteTablet for tablet 3b686a92fe194cd6b2e3d91024d257f3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 05:02:16 UTC) from {username='kudu', principal='kudu/127.9.96.126@KRBTEST.COM'} at 127.0.0.1:50524
I20260806 05:02:16.254272 9601 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu pid 3192
I20260806 05:02:16.258189 3854 tablet_replica.cc:330] T 3b686a92fe194cd6b2e3d91024d257f3 P cce539c2ff6c4af1ab6c74e523fbc3cc: stopping tablet replica
I20260806 05:02:16.258622 3853 tablet_replica.cc:330] T 3b686a92fe194cd6b2e3d91024d257f3 P 62bb147239a74aab8c7fe7f7f90a757e: stopping tablet replica
I20260806 05:02:16.258939 3854 raft_consensus.cc:2229] T 3b686a92fe194cd6b2e3d91024d257f3 P cce539c2ff6c4af1ab6c74e523fbc3cc [term 1 LEADER]: Raft consensus shutting down.
I20260806 05:02:16.259214 3853 raft_consensus.cc:2229] T 3b686a92fe194cd6b2e3d91024d257f3 P 62bb147239a74aab8c7fe7f7f90a757e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 05:02:16.259765 3854 raft_consensus.cc:2258] T 3b686a92fe194cd6b2e3d91024d257f3 P cce539c2ff6c4af1ab6c74e523fbc3cc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 05:02:16.259871 3853 raft_consensus.cc:2258] T 3b686a92fe194cd6b2e3d91024d257f3 P 62bb147239a74aab8c7fe7f7f90a757e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 05:02:16.261034 3852 tablet_replica.cc:330] T 3b686a92fe194cd6b2e3d91024d257f3 P 6b2713f209924713a4e746ced443630e: stopping tablet replica
I20260806 05:02:16.261436 3852 raft_consensus.cc:2229] T 3b686a92fe194cd6b2e3d91024d257f3 P 6b2713f209924713a4e746ced443630e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 05:02:16.261941 3852 raft_consensus.cc:2258] T 3b686a92fe194cd6b2e3d91024d257f3 P 6b2713f209924713a4e746ced443630e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 05:02:16.262410 3853 ts_tablet_manager.cc:1905] T 3b686a92fe194cd6b2e3d91024d257f3 P 62bb147239a74aab8c7fe7f7f90a757e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 05:02:16.263433 3854 ts_tablet_manager.cc:1905] T 3b686a92fe194cd6b2e3d91024d257f3 P cce539c2ff6c4af1ab6c74e523fbc3cc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 05:02:16.263849 3852 ts_tablet_manager.cc:1905] T 3b686a92fe194cd6b2e3d91024d257f3 P 6b2713f209924713a4e746ced443630e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 05:02:16.269634 3853 ts_tablet_manager.cc:1918] T 3b686a92fe194cd6b2e3d91024d257f3 P 62bb147239a74aab8c7fe7f7f90a757e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 05:02:16.269797 3853 log.cc:1196] T 3b686a92fe194cd6b2e3d91024d257f3 P 62bb147239a74aab8c7fe7f7f90a757e: Deleting WAL directory at /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785992227050689-9601-0/minicluster-data/ts-0/wal/wals/3b686a92fe194cd6b2e3d91024d257f3
I20260806 05:02:16.270354 3853 ts_tablet_manager.cc:1939] T 3b686a92fe194cd6b2e3d91024d257f3 P 62bb147239a74aab8c7fe7f7f90a757e: Deleting consensus metadata
I20260806 05:02:16.271440 3852 ts_tablet_manager.cc:1918] T 3b686a92fe194cd6b2e3d91024d257f3 P 6b2713f209924713a4e746ced443630e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 05:02:16.271467 3854 ts_tablet_manager.cc:1918] T 3b686a92fe194cd6b2e3d91024d257f3 P cce539c2ff6c4af1ab6c74e523fbc3cc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 05:02:16.271649 3854 log.cc:1196] T 3b686a92fe194cd6b2e3d91024d257f3 P cce539c2ff6c4af1ab6c74e523fbc3cc: Deleting WAL directory at /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785992227050689-9601-0/minicluster-data/ts-2/wal/wals/3b686a92fe194cd6b2e3d91024d257f3
I20260806 05:02:16.271705 3852 log.cc:1196] T 3b686a92fe194cd6b2e3d91024d257f3 P 6b2713f209924713a4e746ced443630e: Deleting WAL directory at /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785992227050689-9601-0/minicluster-data/ts-1/wal/wals/3b686a92fe194cd6b2e3d91024d257f3
I20260806 05:02:16.272226 3854 ts_tablet_manager.cc:1939] T 3b686a92fe194cd6b2e3d91024d257f3 P cce539c2ff6c4af1ab6c74e523fbc3cc: Deleting consensus metadata
I20260806 05:02:16.272279 3079 catalog_manager.cc:4841] TS 62bb147239a74aab8c7fe7f7f90a757e (127.9.96.65:39517): tablet 3b686a92fe194cd6b2e3d91024d257f3 (table test-table [id=f72282d6d50e437b9cbf15986bcf6c33]) successfully deleted
I20260806 05:02:16.272496 3852 ts_tablet_manager.cc:1939] T 3b686a92fe194cd6b2e3d91024d257f3 P 6b2713f209924713a4e746ced443630e: Deleting consensus metadata
I20260806 05:02:16.273903 3078 catalog_manager.cc:4841] TS 6b2713f209924713a4e746ced443630e (127.9.96.66:42971): tablet 3b686a92fe194cd6b2e3d91024d257f3 (table test-table [id=f72282d6d50e437b9cbf15986bcf6c33]) successfully deleted
I20260806 05:02:16.274145 3078 catalog_manager.cc:4841] TS cce539c2ff6c4af1ab6c74e523fbc3cc (127.9.96.67:36307): tablet 3b686a92fe194cd6b2e3d91024d257f3 (table test-table [id=f72282d6d50e437b9cbf15986bcf6c33]) successfully deleted
Aug 06 05:02:16 dist-test-slave-x4qh krb5kdc[3029](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992533, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.96.65@KRBTEST.COM
I20260806 05:02:16.290747 3807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:02:16.257980 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36958 (local address 127.9.96.65:39517)
0806 05:02:16.258176 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0806 05:02:16.258185 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 05:02:16.272046 (+ 13861us) server_negotiation.cc:407] Connection header received
0806 05:02:16.274002 (+ 1956us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 05:02:16.274010 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 05:02:16.274105 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 05:02:16.274217 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0806 05:02:16.276123 (+ 1906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:02:16.276870 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:02:16.278856 (+ 1986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:02:16.279280 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:02:16.279456 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:02:16.284553 (+ 5097us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 05:02:16.284565 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 05:02:16.284580 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 05:02:16.284612 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0806 05:02:16.287783 (+ 3171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 05:02:16.288546 (+ 763us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 05:02:16.288557 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 05:02:16.288560 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 05:02:16.288737 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 05:02:16.289121 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 05:02:16.289127 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 05:02:16.289130 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 05:02:16.289614 (+ 484us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 05:02:16.289934 (+ 320us) server_negotiation.cc:1035] Waiting for connection context
0806 05:02:16.290107 (+ 173us) server_negotiation.cc:299] Negotiation successful
0806 05:02:16.290357 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260806 05:02:16.291477 3363 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 05:02:16.404872 9601 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu with pid 3192
I20260806 05:02:16.429643 9601 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu pid 3377
Aug 06 05:02:16 dist-test-slave-x4qh krb5kdc[3029](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992533, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.96.66@KRBTEST.COM
I20260806 05:02:16.447007 3804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:02:16.430527 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59234 (local address 127.9.96.66:42971)
0806 05:02:16.430733 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0806 05:02:16.430739 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 05:02:16.430763 (+ 24us) server_negotiation.cc:407] Connection header received
0806 05:02:16.430949 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 05:02:16.430954 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 05:02:16.431071 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 05:02:16.431187 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0806 05:02:16.432709 (+ 1522us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:02:16.433830 (+ 1121us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:02:16.435292 (+ 1462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:02:16.435944 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:02:16.436074 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:02:16.440228 (+ 4154us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 05:02:16.440246 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 05:02:16.440267 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 05:02:16.440309 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0806 05:02:16.444116 (+ 3807us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 05:02:16.444809 (+ 693us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 05:02:16.444819 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 05:02:16.444823 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 05:02:16.444987 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 05:02:16.445383 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 05:02:16.445389 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 05:02:16.445393 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 05:02:16.445887 (+ 494us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 05:02:16.446223 (+ 336us) server_negotiation.cc:1035] Waiting for connection context
0806 05:02:16.446388 (+ 165us) server_negotiation.cc:299] Negotiation successful
0806 05:02:16.446601 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260806 05:02:16.447856 3560 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 05:02:16.560156 9601 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu with pid 3377
I20260806 05:02:16.585981 9601 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu pid 3578
Aug 06 05:02:16 dist-test-slave-x4qh krb5kdc[3029](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992533, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.96.67@KRBTEST.COM
I20260806 05:02:16.604890 3810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:02:16.586946 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50544 (local address 127.9.96.67:36307)
0806 05:02:16.587225 (+ 279us) server_negotiation.cc:206] Beginning negotiation
0806 05:02:16.587231 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 05:02:16.587256 (+ 25us) server_negotiation.cc:407] Connection header received
0806 05:02:16.587561 (+ 305us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 05:02:16.587567 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 05:02:16.587660 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 05:02:16.587804 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0806 05:02:16.589141 (+ 1337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:02:16.590333 (+ 1192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:02:16.591679 (+ 1346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:02:16.592455 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:02:16.592609 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:02:16.597308 (+ 4699us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 05:02:16.597328 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 05:02:16.597351 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 05:02:16.597396 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0806 05:02:16.601698 (+ 4302us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 05:02:16.602482 (+ 784us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 05:02:16.602492 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 05:02:16.602495 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 05:02:16.602671 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 05:02:16.603152 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 05:02:16.603158 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 05:02:16.603167 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0806 05:02:16.603738 (+ 571us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 05:02:16.604056 (+ 318us) server_negotiation.cc:1035] Waiting for connection context
0806 05:02:16.604245 (+ 189us) server_negotiation.cc:299] Negotiation successful
0806 05:02:16.604517 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260806 05:02:16.605723 3770 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 05:02:16.734170 9601 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu with pid 3578
I20260806 05:02:16.760825 9601 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu pid 3045
I20260806 05:02:16.762048 3150 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 05:02:16.871430 9601 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu with pid 3045
2026-08-06T05:02:16Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (5011 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-08-06T05:02:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T05:02:16Z Disabled control of system clock
I20260806 05:02:16.964238 9601 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
/tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785992227050689-9601-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785992227050689-9601-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785992227050689-9601-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785992227050689-9601-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.96.126:41613
--webserver_interface=127.9.96.126
--webserver_port=0
--builtin_ntp_servers=127.9.96.84:44121
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.96.126:41613
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260806 05:02:17.300526 3864 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 05:02:17.300828 3864 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 05:02:17.300917 3864 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 05:02:17.308127 3864 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 05:02:17.308244 3864 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 05:02:17.308281 3864 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 05:02:17.308331 3864 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 05:02:17.308375 3864 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 05:02:17.317472 3864 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.96.84:44121
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785992227050689-9601-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785992227050689-9601-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.96.126:41613
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.96.126:41613
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785992227050689-9601-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.96.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785992227050689-9601-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 03:56:12 UTC on ccycloud.quasar-guijbm.root.comops.site
build id 4569618
ASAN enabled
I20260806 05:02:17.318804 3864 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 05:02:17.320643 3864 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 05:02:17.332907 3870 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 05:02:17.332907 3874 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 05:02:17.332960 3872 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 05:02:17.334079 3864 server_base.cc:1034] running on GCE node
I20260806 05:02:17.334873 3864 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 05:02:17.336374 3864 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 05:02:17.337615 3864 hybrid_clock.cc:648] HybridClock initialized: now 1785992537337569 us; error 67 us; skew 500 ppm
I20260806 05:02:17.337975 3864 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 05:02:17.340422 3864 webserver.cc:460] Webserver started at http://127.9.96.126:41095/ using document root <none> and password file <none>
I20260806 05:02:17.341131 3864 fs_manager.cc:359] Metadata directory not provided
I20260806 05:02:17.341267 3864 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 05:02:17.341601 3864 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 05:02:17.344651 3864 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785992227050689-9601-0/minicluster-data/master-0/data/instance:
uuid: "65d2bdeb1989438991a5c4a5c90d8089"
format_stamp: "Formatted at 2026-08-06 05:02:17 on dist-test-slave-x4qh"
server_key: "ff9d2a304bde905c140e3f78cadd12b2"
server_key_iv: "be792244b7b23f140553f12b3622a7ca"
server_key_version: "encryptionkey@0"
I20260806 05:02:17.345492 3864 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785992227050689-9601-0/minicluster-data/master-0/wal/instance:
uuid: "65d2bdeb1989438991a5c4a5c90d8089"
format_stamp: "Formatted at 2026-08-06 05:02:17 on dist-test-slave-x4qh"
server_key: "ff9d2a304bde905c140e3f78cadd12b2"
server_key_iv: "be792244b7b23f140553f12b3622a7ca"
server_key_version: "encryptionkey@0"
I20260806 05:02:17.350816 3864 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.005s
I20260806 05:02:17.354384 3882 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 05:02:17.355842 3864 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 05:02:17.356036 3864 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785992227050689-9601-0/minicluster-data/master-0/data,/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785992227050689-9601-0/minicluster-data/master-0/wal
uuid: "65d2bdeb1989438991a5c4a5c90d8089"
format_stamp: "Formatted at 2026-08-06 05:02:17 on dist-test-slave-x4qh"
server_key: "ff9d2a304bde905c140e3f78cadd12b2"
server_key_iv: "be792244b7b23f140553f12b3622a7ca"
server_key_version: "encryptionkey@0"
I20260806 05:02:17.356267 3864 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785992227050689-9601-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785992227050689-9601-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785992227050689-9601-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 05:02:17.389647 3864 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 05:02:17.390846 3864 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 05:02:17.391204 3864 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 05:02:17.416829 3864 rpc_server.cc:307] RPC server started. Bound to: 127.9.96.126:41613
I20260806 05:02:17.416865 3958 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.96.126:41613 every 8 connection(s)
I20260806 05:02:17.418676 3864 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785992227050689-9601-0/minicluster-data/master-0/data/info.pb
I20260806 05:02:17.421473 9601 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu as pid 3864
I20260806 05:02:17.421665 9601 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785992227050689-9601-0/minicluster-data/master-0/wal/instance
I20260806 05:02:17.422276 9601 external_mini_cluster.cc:1428] Setting key d5b7001a61f4ba763e241552e0f73898
I20260806 05:02:17.423768 3959 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 05:02:17.443055 3959 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 65d2bdeb1989438991a5c4a5c90d8089: Bootstrap starting.
I20260806 05:02:17.445708 3963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:02:17.424255 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47994 (local address 127.9.96.126:41613)
0806 05:02:17.432112 (+ 7857us) server_negotiation.cc:206] Beginning negotiation
0806 05:02:17.432122 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 05:02:17.432165 (+ 43us) server_negotiation.cc:407] Connection header received
0806 05:02:17.433815 (+ 1650us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 05:02:17.433838 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 05:02:17.434287 (+ 449us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 05:02:17.434614 (+ 327us) server_negotiation.cc:226] Negotiated authn=SASL
0806 05:02:17.435465 (+ 851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:02:17.437570 (+ 2105us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:02:17.439952 (+ 2382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:02:17.441056 (+ 1104us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:02:17.441234 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:02:17.442731 (+ 1497us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 05:02:17.442743 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 05:02:17.442760 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 05:02:17.443524 (+ 764us) server_negotiation.cc:922] calling sasl_server_start()
0806 05:02:17.443629 (+ 105us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 05:02:17.443661 (+ 32us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 05:02:17.443787 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0806 05:02:17.444033 (+ 246us) server_negotiation.cc:299] Negotiation successful
0806 05:02:17.444354 (+ 321us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7445,"thread_start_us":7221,"threads_started":1}
I20260806 05:02:17.446943 3959 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 65d2bdeb1989438991a5c4a5c90d8089: Neither blocks nor log segments found. Creating new log.
I20260806 05:02:17.448370 3959 log.cc:824] T 00000000000000000000000000000000 P 65d2bdeb1989438991a5c4a5c90d8089: Log is configured to *not* fsync() on all Append() calls
I20260806 05:02:17.451117 3959 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 65d2bdeb1989438991a5c4a5c90d8089: No bootstrap required, opened a new log
I20260806 05:02:17.458947 3959 raft_consensus.cc:348] T 00000000000000000000000000000000 P 65d2bdeb1989438991a5c4a5c90d8089 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "65d2bdeb1989438991a5c4a5c90d8089" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 41613 } }
I20260806 05:02:17.459268 3959 raft_consensus.cc:374] T 00000000000000000000000000000000 P 65d2bdeb1989438991a5c4a5c90d8089 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 05:02:17.459350 3959 raft_consensus.cc:729] T 00000000000000000000000000000000 P 65d2bdeb1989438991a5c4a5c90d8089 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 65d2bdeb1989438991a5c4a5c90d8089, State: Initialized, Role: FOLLOWER
I20260806 05:02:17.459944 3959 consensus_queue.cc:260] T 00000000000000000000000000000000 P 65d2bdeb1989438991a5c4a5c90d8089 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "65d2bdeb1989438991a5c4a5c90d8089" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 41613 } }
I20260806 05:02:17.460145 3959 raft_consensus.cc:388] T 00000000000000000000000000000000 P 65d2bdeb1989438991a5c4a5c90d8089 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 05:02:17.460237 3959 raft_consensus.cc:482] T 00000000000000000000000000000000 P 65d2bdeb1989438991a5c4a5c90d8089 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 05:02:17.460361 3959 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 65d2bdeb1989438991a5c4a5c90d8089 [term 0 FOLLOWER]: Advancing to term 1
I20260806 05:02:17.462532 3959 raft_consensus.cc:504] T 00000000000000000000000000000000 P 65d2bdeb1989438991a5c4a5c90d8089 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "65d2bdeb1989438991a5c4a5c90d8089" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 41613 } }
I20260806 05:02:17.462955 3959 leader_election.cc:302] T 00000000000000000000000000000000 P 65d2bdeb1989438991a5c4a5c90d8089 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 65d2bdeb1989438991a5c4a5c90d8089; no voters:
I20260806 05:02:17.463553 3959 leader_election.cc:288] T 00000000000000000000000000000000 P 65d2bdeb1989438991a5c4a5c90d8089 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 05:02:17.463681 3967 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 65d2bdeb1989438991a5c4a5c90d8089 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 05:02:17.464448 3967 raft_consensus.cc:686] T 00000000000000000000000000000000 P 65d2bdeb1989438991a5c4a5c90d8089 [term 1 LEADER]: Becoming Leader. State: Replica: 65d2bdeb1989438991a5c4a5c90d8089, State: Running, Role: LEADER
I20260806 05:02:17.464977 3967 consensus_queue.cc:237] T 00000000000000000000000000000000 P 65d2bdeb1989438991a5c4a5c90d8089 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "65d2bdeb1989438991a5c4a5c90d8089" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 41613 } }
I20260806 05:02:17.465791 3959 sys_catalog.cc:564] T 00000000000000000000000000000000 P 65d2bdeb1989438991a5c4a5c90d8089 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 05:02:17.467999 3968 sys_catalog.cc:455] T 00000000000000000000000000000000 P 65d2bdeb1989438991a5c4a5c90d8089 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "65d2bdeb1989438991a5c4a5c90d8089" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "65d2bdeb1989438991a5c4a5c90d8089" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 41613 } } }
I20260806 05:02:17.468519 3968 sys_catalog.cc:458] T 00000000000000000000000000000000 P 65d2bdeb1989438991a5c4a5c90d8089 [sys.catalog]: This master's current role is: LEADER
I20260806 05:02:17.467993 3969 sys_catalog.cc:455] T 00000000000000000000000000000000 P 65d2bdeb1989438991a5c4a5c90d8089 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 65d2bdeb1989438991a5c4a5c90d8089. Latest consensus state: current_term: 1 leader_uuid: "65d2bdeb1989438991a5c4a5c90d8089" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "65d2bdeb1989438991a5c4a5c90d8089" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 41613 } } }
I20260806 05:02:17.469033 3969 sys_catalog.cc:458] T 00000000000000000000000000000000 P 65d2bdeb1989438991a5c4a5c90d8089 [sys.catalog]: This master's current role is: LEADER
I20260806 05:02:17.471083 3973 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 05:02:17.475409 3973 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 05:02:17.484287 3973 catalog_manager.cc:1347] Generated new cluster ID: c0993d96a83740aabdb5ce76d2c10380
I20260806 05:02:17.484407 3973 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 05:02:17.514011 3973 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 05:02:17.515543 3973 catalog_manager.cc:1504] Loading token signing keys...
I20260806 05:02:17.523186 3973 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 65d2bdeb1989438991a5c4a5c90d8089: Generated new TSK 0
I20260806 05:02:17.524017 3973 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 05:02:17.539306 9601 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
/tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785992227050689-9601-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785992227050689-9601-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785992227050689-9601-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785992227050689-9601-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.96.65:0
--local_ip_for_outbound_sockets=127.9.96.65
--webserver_interface=127.9.96.65
--webserver_port=0
--tserver_master_addrs=127.9.96.126:41613
--builtin_ntp_servers=127.9.96.84:44121
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260806 05:02:17.879999 4000 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 05:02:17.880265 4000 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 05:02:17.880437 4000 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 05:02:17.887691 4000 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 05:02:17.887784 4000 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 05:02:17.887953 4000 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.96.65
I20260806 05:02:17.897126 4000 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.96.84:44121
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785992227050689-9601-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785992227050689-9601-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.96.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785992227050689-9601-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.96.65
--webserver_port=0
--tserver_master_addrs=127.9.96.126:41613
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.96.65
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785992227050689-9601-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 03:56:12 UTC on ccycloud.quasar-guijbm.root.comops.site
build id 4569618
ASAN enabled
I20260806 05:02:17.898407 4000 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 05:02:17.900236 4000 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 05:02:17.913260 4007 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 05:02:17.913184 4008 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 05:02:17.913295 4012 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 05:02:17.914292 4000 server_base.cc:1034] running on GCE node
I20260806 05:02:17.914947 4000 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 05:02:17.915971 4000 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 05:02:17.917215 4000 hybrid_clock.cc:648] HybridClock initialized: now 1785992537917171 us; error 57 us; skew 500 ppm
I20260806 05:02:17.917541 4000 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 05:02:17.919858 4000 webserver.cc:460] Webserver started at http://127.9.96.65:39291/ using document root <none> and password file <none>
I20260806 05:02:17.920555 4000 fs_manager.cc:359] Metadata directory not provided
I20260806 05:02:17.920709 4000 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 05:02:17.921053 4000 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 05:02:17.923965 4000 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785992227050689-9601-0/minicluster-data/ts-0/data/instance:
uuid: "b2e228824a4a4af9a0aa32913c0cbf2a"
format_stamp: "Formatted at 2026-08-06 05:02:17 on dist-test-slave-x4qh"
server_key: "6258563a431bf5b6a0a1db25ce38e00d"
server_key_iv: "f97f3ee15f5f71326d070784499455bc"
server_key_version: "encryptionkey@0"
I20260806 05:02:17.924799 4000 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785992227050689-9601-0/minicluster-data/ts-0/wal/instance:
uuid: "b2e228824a4a4af9a0aa32913c0cbf2a"
format_stamp: "Formatted at 2026-08-06 05:02:17 on dist-test-slave-x4qh"
server_key: "6258563a431bf5b6a0a1db25ce38e00d"
server_key_iv: "f97f3ee15f5f71326d070784499455bc"
server_key_version: "encryptionkey@0"
I20260806 05:02:17.929956 4000 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260806 05:02:17.933468 4020 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 05:02:17.934760 4000 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 05:02:17.934938 4000 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785992227050689-9601-0/minicluster-data/ts-0/data,/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785992227050689-9601-0/minicluster-data/ts-0/wal
uuid: "b2e228824a4a4af9a0aa32913c0cbf2a"
format_stamp: "Formatted at 2026-08-06 05:02:17 on dist-test-slave-x4qh"
server_key: "6258563a431bf5b6a0a1db25ce38e00d"
server_key_iv: "f97f3ee15f5f71326d070784499455bc"
server_key_version: "encryptionkey@0"
I20260806 05:02:17.935173 4000 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785992227050689-9601-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785992227050689-9601-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785992227050689-9601-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 05:02:17.955322 4000 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 05:02:17.956177 4000 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 05:02:17.956468 4000 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 05:02:17.957419 4000 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 05:02:17.959144 4000 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 05:02:17.959271 4000 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 05:02:17.959425 4000 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 05:02:17.959527 4000 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 05:02:17.993991 4000 rpc_server.cc:307] RPC server started. Bound to: 127.9.96.65:36897
I20260806 05:02:17.994078 4177 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.96.65:36897 every 8 connection(s)
I20260806 05:02:17.995282 4000 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785992227050689-9601-0/minicluster-data/ts-0/data/info.pb
I20260806 05:02:17.999457 9601 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu as pid 4000
I20260806 05:02:17.999681 9601 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785992227050689-9601-0/minicluster-data/ts-0/wal/instance
I20260806 05:02:18.000149 9601 external_mini_cluster.cc:1428] Setting key 48727c106931df9c8a8bf10fe412ca27
I20260806 05:02:18.002907 9601 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
/tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785992227050689-9601-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785992227050689-9601-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785992227050689-9601-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785992227050689-9601-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.96.66:0
--local_ip_for_outbound_sockets=127.9.96.66
--webserver_interface=127.9.96.66
--webserver_port=0
--tserver_master_addrs=127.9.96.126:41613
--builtin_ntp_servers=127.9.96.84:44121
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260806 05:02:18.008189 4181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:02:17.997858 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.96.65:46011 (local address 127.9.96.126:41613)
0806 05:02:17.998324 (+ 466us) server_negotiation.cc:206] Beginning negotiation
0806 05:02:17.998330 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 05:02:17.999557 (+ 1227us) server_negotiation.cc:407] Connection header received
0806 05:02:18.001599 (+ 2042us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 05:02:18.001605 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 05:02:18.001699 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 05:02:18.001821 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0806 05:02:18.002902 (+ 1081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:02:18.003787 (+ 885us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:02:18.005374 (+ 1587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:02:18.005774 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:02:18.005902 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:02:18.007294 (+ 1392us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 05:02:18.007299 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 05:02:18.007302 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 05:02:18.007319 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0806 05:02:18.007358 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 05:02:18.007418 (+ 60us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 05:02:18.007524 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0806 05:02:18.007757 (+ 233us) server_negotiation.cc:299] Negotiation successful
0806 05:02:18.007855 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":336,"thread_start_us":222,"threads_started":1}
I20260806 05:02:18.009152 4182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:02:17.998450 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:41613 (local address 127.9.96.65:46011)
0806 05:02:17.999354 (+ 904us) negotiation.cc:107] Waiting for socket to connect
0806 05:02:17.999441 (+ 87us) client_negotiation.cc:174] Beginning negotiation
0806 05:02:18.001342 (+ 1901us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 05:02:18.002058 (+ 716us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 05:02:18.002066 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 05:02:18.002151 (+ 85us) client_negotiation.cc:189] Negotiated authn=SASL
0806 05:02:18.002724 (+ 573us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 05:02:18.002741 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 05:02:18.003948 (+ 1207us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 05:02:18.003952 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 05:02:18.005208 (+ 1256us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 05:02:18.005221 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 05:02:18.006003 (+ 782us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 05:02:18.006011 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 05:02:18.006236 (+ 225us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:02:18.006963 (+ 727us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 05:02:18.006966 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0806 05:02:18.007004 (+ 38us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 05:02:18.007082 (+ 78us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 05:02:18.007524 (+ 442us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 05:02:18.007538 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 05:02:18.007548 (+ 10us) client_negotiation.cc:769] Sending connection context
0806 05:02:18.007737 (+ 189us) client_negotiation.cc:240] Negotiation successful
0806 05:02:18.008124 (+ 387us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":472,"thread_start_us":290,"threads_started":1}
I20260806 05:02:18.010743 4178 heartbeater.cc:347] Connected to a master server at 127.9.96.126:41613
I20260806 05:02:18.011039 4178 heartbeater.cc:464] Registering TS with master...
I20260806 05:02:18.011826 4178 heartbeater.cc:511] Master 127.9.96.126:41613 requested a full tablet report, sending...
I20260806 05:02:18.014070 3908 ts_manager.cc:194] Registered new tserver with Master: b2e228824a4a4af9a0aa32913c0cbf2a (127.9.96.65:36897)
I20260806 05:02:18.015630 3908 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.9.96.65:46011
W20260806 05:02:18.338040 4183 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 05:02:18.338363 4183 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 05:02:18.338534 4183 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 05:02:18.345808 4183 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 05:02:18.345902 4183 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 05:02:18.346071 4183 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.96.66
I20260806 05:02:18.355062 4183 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.96.84:44121
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785992227050689-9601-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785992227050689-9601-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.96.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785992227050689-9601-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.96.66
--webserver_port=0
--tserver_master_addrs=127.9.96.126:41613
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.96.66
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785992227050689-9601-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 03:56:12 UTC on ccycloud.quasar-guijbm.root.comops.site
build id 4569618
ASAN enabled
I20260806 05:02:18.356297 4183 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 05:02:18.357976 4183 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 05:02:18.371397 4192 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 05:02:18.371492 4190 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 05:02:18.371704 4189 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 05:02:18.372414 4183 server_base.cc:1034] running on GCE node
I20260806 05:02:18.373018 4183 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 05:02:18.373891 4183 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 05:02:18.375130 4183 hybrid_clock.cc:648] HybridClock initialized: now 1785992538375087 us; error 50 us; skew 500 ppm
I20260806 05:02:18.375535 4183 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 05:02:18.377887 4183 webserver.cc:460] Webserver started at http://127.9.96.66:35919/ using document root <none> and password file <none>
I20260806 05:02:18.378583 4183 fs_manager.cc:359] Metadata directory not provided
I20260806 05:02:18.378727 4183 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 05:02:18.379061 4183 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 05:02:18.381898 4183 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785992227050689-9601-0/minicluster-data/ts-1/data/instance:
uuid: "593860df0812495eb4254fcd36b0a511"
format_stamp: "Formatted at 2026-08-06 05:02:18 on dist-test-slave-x4qh"
server_key: "212bf99ebab7598dd2cc5ce6e8d2cbc8"
server_key_iv: "d0c10ce31dbb6ae291563b6573062e08"
server_key_version: "encryptionkey@0"
I20260806 05:02:18.382699 4183 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785992227050689-9601-0/minicluster-data/ts-1/wal/instance:
uuid: "593860df0812495eb4254fcd36b0a511"
format_stamp: "Formatted at 2026-08-06 05:02:18 on dist-test-slave-x4qh"
server_key: "212bf99ebab7598dd2cc5ce6e8d2cbc8"
server_key_iv: "d0c10ce31dbb6ae291563b6573062e08"
server_key_version: "encryptionkey@0"
I20260806 05:02:18.387730 4183 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.004s
I20260806 05:02:18.391168 4201 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 05:02:18.392449 4183 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 05:02:18.392632 4183 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785992227050689-9601-0/minicluster-data/ts-1/data,/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785992227050689-9601-0/minicluster-data/ts-1/wal
uuid: "593860df0812495eb4254fcd36b0a511"
format_stamp: "Formatted at 2026-08-06 05:02:18 on dist-test-slave-x4qh"
server_key: "212bf99ebab7598dd2cc5ce6e8d2cbc8"
server_key_iv: "d0c10ce31dbb6ae291563b6573062e08"
server_key_version: "encryptionkey@0"
I20260806 05:02:18.392858 4183 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785992227050689-9601-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785992227050689-9601-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785992227050689-9601-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 05:02:18.412034 4183 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 05:02:18.412842 4183 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 05:02:18.413131 4183 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 05:02:18.414079 4183 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 05:02:18.415644 4183 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 05:02:18.415781 4183 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 05:02:18.415921 4183 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 05:02:18.416007 4183 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 05:02:18.452143 4183 rpc_server.cc:307] RPC server started. Bound to: 127.9.96.66:43299
I20260806 05:02:18.452236 4365 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.96.66:43299 every 8 connection(s)
I20260806 05:02:18.453439 4183 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785992227050689-9601-0/minicluster-data/ts-1/data/info.pb
I20260806 05:02:18.463423 9601 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu as pid 4183
I20260806 05:02:18.463614 9601 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785992227050689-9601-0/minicluster-data/ts-1/wal/instance
I20260806 05:02:18.464108 9601 external_mini_cluster.cc:1428] Setting key 0b01d3b4909d73a7f8e676ccc2f8e1e2
I20260806 05:02:18.466279 4181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:02:18.456002 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.96.66:42867 (local address 127.9.96.126:41613)
0806 05:02:18.456234 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0806 05:02:18.456239 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 05:02:18.457596 (+ 1357us) server_negotiation.cc:407] Connection header received
0806 05:02:18.459279 (+ 1683us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 05:02:18.459284 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 05:02:18.459497 (+ 213us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 05:02:18.459616 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0806 05:02:18.460971 (+ 1355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:02:18.461699 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:02:18.463256 (+ 1557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:02:18.463840 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:02:18.463978 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:02:18.465432 (+ 1454us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 05:02:18.465438 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 05:02:18.465442 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 05:02:18.465467 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0806 05:02:18.465501 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 05:02:18.465513 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 05:02:18.465614 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0806 05:02:18.465863 (+ 249us) server_negotiation.cc:299] Negotiation successful
0806 05:02:18.465971 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260806 05:02:18.467090 9601 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
/tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785992227050689-9601-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785992227050689-9601-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785992227050689-9601-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785992227050689-9601-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.96.67:0
--local_ip_for_outbound_sockets=127.9.96.67
--webserver_interface=127.9.96.67
--webserver_port=0
--tserver_master_addrs=127.9.96.126:41613
--builtin_ntp_servers=127.9.96.84:44121
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260806 05:02:18.467407 4369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:02:18.456567 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:41613 (local address 127.9.96.66:42867)
0806 05:02:18.457430 (+ 863us) negotiation.cc:107] Waiting for socket to connect
0806 05:02:18.457483 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0806 05:02:18.459041 (+ 1558us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 05:02:18.459935 (+ 894us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 05:02:18.459943 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 05:02:18.460027 (+ 84us) client_negotiation.cc:189] Negotiated authn=SASL
0806 05:02:18.460773 (+ 746us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 05:02:18.460793 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 05:02:18.461860 (+ 1067us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 05:02:18.461864 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 05:02:18.463087 (+ 1223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 05:02:18.463099 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 05:02:18.464080 (+ 981us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 05:02:18.464087 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 05:02:18.464369 (+ 282us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:02:18.465109 (+ 740us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 05:02:18.465111 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0806 05:02:18.465155 (+ 44us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 05:02:18.465231 (+ 76us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 05:02:18.465613 (+ 382us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 05:02:18.465627 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 05:02:18.465637 (+ 10us) client_negotiation.cc:769] Sending connection context
0806 05:02:18.465832 (+ 195us) client_negotiation.cc:240] Negotiation successful
0806 05:02:18.466215 (+ 383us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":470,"thread_start_us":289,"threads_started":1}
I20260806 05:02:18.468757 4366 heartbeater.cc:347] Connected to a master server at 127.9.96.126:41613
I20260806 05:02:18.469071 4366 heartbeater.cc:464] Registering TS with master...
I20260806 05:02:18.469784 4366 heartbeater.cc:511] Master 127.9.96.126:41613 requested a full tablet report, sending...
I20260806 05:02:18.471277 3908 ts_manager.cc:194] Registered new tserver with Master: 593860df0812495eb4254fcd36b0a511 (127.9.96.66:43299)
I20260806 05:02:18.472465 3908 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.9.96.66:42867
W20260806 05:02:18.804302 4370 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 05:02:18.804593 4370 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 05:02:18.804680 4370 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 05:02:18.811771 4370 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 05:02:18.811869 4370 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 05:02:18.811971 4370 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.96.67
I20260806 05:02:18.821026 4370 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.96.84:44121
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785992227050689-9601-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785992227050689-9601-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.96.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785992227050689-9601-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.96.67
--webserver_port=0
--tserver_master_addrs=127.9.96.126:41613
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.96.67
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785992227050689-9601-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 03:56:12 UTC on ccycloud.quasar-guijbm.root.comops.site
build id 4569618
ASAN enabled
I20260806 05:02:18.822278 4370 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 05:02:18.823994 4370 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 05:02:18.837154 4378 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 05:02:18.837121 4380 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 05:02:18.837559 4384 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 05:02:18.838433 4370 server_base.cc:1034] running on GCE node
I20260806 05:02:18.839102 4370 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 05:02:18.840155 4370 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 05:02:18.841355 4370 hybrid_clock.cc:648] HybridClock initialized: now 1785992538841301 us; error 48 us; skew 500 ppm
I20260806 05:02:18.841657 4370 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 05:02:18.843859 4370 webserver.cc:460] Webserver started at http://127.9.96.67:43299/ using document root <none> and password file <none>
I20260806 05:02:18.844538 4370 fs_manager.cc:359] Metadata directory not provided
I20260806 05:02:18.844657 4370 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 05:02:18.845002 4370 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 05:02:18.847863 4370 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785992227050689-9601-0/minicluster-data/ts-2/data/instance:
uuid: "1579aec8cc7f46a8a4260aa3e4e942e3"
format_stamp: "Formatted at 2026-08-06 05:02:18 on dist-test-slave-x4qh"
server_key: "729e75bdfc9bb28c31e242bdcd7d0f96"
server_key_iv: "87e8b5a47810b83636534f786401895a"
server_key_version: "encryptionkey@0"
I20260806 05:02:18.848691 4370 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785992227050689-9601-0/minicluster-data/ts-2/wal/instance:
uuid: "1579aec8cc7f46a8a4260aa3e4e942e3"
format_stamp: "Formatted at 2026-08-06 05:02:18 on dist-test-slave-x4qh"
server_key: "729e75bdfc9bb28c31e242bdcd7d0f96"
server_key_iv: "87e8b5a47810b83636534f786401895a"
server_key_version: "encryptionkey@0"
I20260806 05:02:18.853543 4370 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.003s
I20260806 05:02:18.856917 4391 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 05:02:18.858099 4370 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260806 05:02:18.858301 4370 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785992227050689-9601-0/minicluster-data/ts-2/data,/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785992227050689-9601-0/minicluster-data/ts-2/wal
uuid: "1579aec8cc7f46a8a4260aa3e4e942e3"
format_stamp: "Formatted at 2026-08-06 05:02:18 on dist-test-slave-x4qh"
server_key: "729e75bdfc9bb28c31e242bdcd7d0f96"
server_key_iv: "87e8b5a47810b83636534f786401895a"
server_key_version: "encryptionkey@0"
I20260806 05:02:18.858530 4370 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785992227050689-9601-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785992227050689-9601-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785992227050689-9601-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 05:02:18.876842 4370 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 05:02:18.877612 4370 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 05:02:18.877914 4370 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 05:02:18.878861 4370 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 05:02:18.880390 4370 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 05:02:18.880517 4370 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 05:02:18.880654 4370 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 05:02:18.880744 4370 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 05:02:18.915812 4370 rpc_server.cc:307] RPC server started. Bound to: 127.9.96.67:33743
I20260806 05:02:18.915910 4559 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.96.67:33743 every 8 connection(s)
I20260806 05:02:18.917161 4370 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785992227050689-9601-0/minicluster-data/ts-2/data/info.pb
I20260806 05:02:18.917512 9601 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu as pid 4370
I20260806 05:02:18.917718 9601 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785992227050689-9601-0/minicluster-data/ts-2/wal/instance
I20260806 05:02:18.918378 9601 external_mini_cluster.cc:1428] Setting key 58b45f97d6b198a61bc86897e75725bc
I20260806 05:02:18.933966 4181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:02:18.919966 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.96.67:53761 (local address 127.9.96.126:41613)
0806 05:02:18.920197 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0806 05:02:18.920204 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 05:02:18.921996 (+ 1792us) server_negotiation.cc:407] Connection header received
0806 05:02:18.923861 (+ 1865us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 05:02:18.923866 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 05:02:18.923961 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 05:02:18.924074 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0806 05:02:18.925141 (+ 1067us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:02:18.925892 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:02:18.927502 (+ 1610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:02:18.927917 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:02:18.928073 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:02:18.933042 (+ 4969us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 05:02:18.933047 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 05:02:18.933050 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 05:02:18.933068 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0806 05:02:18.933104 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 05:02:18.933119 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 05:02:18.933223 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0806 05:02:18.933474 (+ 251us) server_negotiation.cc:299] Negotiation successful
0806 05:02:18.933578 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260806 05:02:18.934921 4563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:02:18.920696 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:41613 (local address 127.9.96.67:53761)
0806 05:02:18.921848 (+ 1152us) negotiation.cc:107] Waiting for socket to connect
0806 05:02:18.921892 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0806 05:02:18.923590 (+ 1698us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 05:02:18.924324 (+ 734us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 05:02:18.924332 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 05:02:18.924428 (+ 96us) client_negotiation.cc:189] Negotiated authn=SASL
0806 05:02:18.924971 (+ 543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 05:02:18.924988 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 05:02:18.926068 (+ 1080us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 05:02:18.926073 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 05:02:18.927294 (+ 1221us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 05:02:18.927307 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 05:02:18.928157 (+ 850us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 05:02:18.928164 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 05:02:18.928395 (+ 231us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:02:18.932694 (+ 4299us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 05:02:18.932697 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0806 05:02:18.932738 (+ 41us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 05:02:18.932831 (+ 93us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 05:02:18.933224 (+ 393us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 05:02:18.933237 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 05:02:18.933262 (+ 25us) client_negotiation.cc:769] Sending connection context
0806 05:02:18.933442 (+ 180us) client_negotiation.cc:240] Negotiation successful
0806 05:02:18.933842 (+ 400us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":643,"thread_start_us":419,"threads_started":1}
I20260806 05:02:18.936316 4561 heartbeater.cc:347] Connected to a master server at 127.9.96.126:41613
I20260806 05:02:18.936615 4561 heartbeater.cc:464] Registering TS with master...
I20260806 05:02:18.937351 4561 heartbeater.cc:511] Master 127.9.96.126:41613 requested a full tablet report, sending...
I20260806 05:02:18.938746 3908 ts_manager.cc:194] Registered new tserver with Master: 1579aec8cc7f46a8a4260aa3e4e942e3 (127.9.96.67:33743)
I20260806 05:02:18.939714 3908 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.9.96.67:53761
I20260806 05:02:18.946031 9601 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 05:02:18.955582 4181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:02:18.950009 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47998 (local address 127.9.96.126:41613)
0806 05:02:18.950254 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0806 05:02:18.950260 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 05:02:18.950509 (+ 249us) server_negotiation.cc:407] Connection header received
0806 05:02:18.950632 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 05:02:18.950635 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 05:02:18.950703 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 05:02:18.950801 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0806 05:02:18.951216 (+ 415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:02:18.951984 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:02:18.953431 (+ 1447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:02:18.953850 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:02:18.953941 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:02:18.954901 (+ 960us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 05:02:18.954906 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 05:02:18.954909 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 05:02:18.954925 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0806 05:02:18.954955 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 05:02:18.954967 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 05:02:18.955061 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0806 05:02:18.955142 (+ 81us) server_negotiation.cc:299] Negotiation successful
0806 05:02:18.955231 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260806 05:02:18.959112 3908 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47998:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 05:02:18.966914 3908 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 05:02:19.000349 4576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:02:18.978621 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.67:33743 (local address 127.0.0.1:48756)
0806 05:02:18.980217 (+ 1596us) negotiation.cc:107] Waiting for socket to connect
0806 05:02:18.980236 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0806 05:02:18.980373 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 05:02:18.991142 (+ 10769us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 05:02:18.991146 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 05:02:18.991174 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 05:02:18.992538 (+ 1364us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 05:02:18.992560 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 05:02:18.994503 (+ 1943us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 05:02:18.994508 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 05:02:18.996827 (+ 2319us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 05:02:18.996850 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 05:02:18.999187 (+ 2337us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 05:02:18.999192 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 05:02:18.999523 (+ 331us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:02:18.999594 (+ 71us) client_negotiation.cc:769] Sending connection context
0806 05:02:18.999720 (+ 126us) client_negotiation.cc:240] Negotiation successful
0806 05:02:18.999930 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1462,"mutex_wait_us":83,"spinlock_wait_cycles":5504,"thread_start_us":2497,"threads_started":1}
I20260806 05:02:19.000388 4577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:02:18.978821 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57548 (local address 127.9.96.66:43299)
0806 05:02:18.990524 (+ 11703us) server_negotiation.cc:206] Beginning negotiation
0806 05:02:18.990541 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0806 05:02:18.990566 (+ 25us) server_negotiation.cc:407] Connection header received
0806 05:02:18.990650 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 05:02:18.990657 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 05:02:18.990872 (+ 215us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 05:02:18.991287 (+ 415us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 05:02:18.992731 (+ 1444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:02:18.994100 (+ 1369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:02:18.997113 (+ 3013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:02:18.998595 (+ 1482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:02:18.998763 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:02:18.999009 (+ 246us) server_negotiation.cc:1035] Waiting for connection context
0806 05:02:18.999901 (+ 892us) server_negotiation.cc:299] Negotiation successful
0806 05:02:19.000067 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11546,"thread_start_us":12075,"threads_started":1}
I20260806 05:02:19.001492 4574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:02:18.978458 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.66:43299 (local address 127.0.0.1:57548)
0806 05:02:18.979445 (+ 987us) negotiation.cc:107] Waiting for socket to connect
0806 05:02:18.979493 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0806 05:02:18.979681 (+ 188us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 05:02:18.991117 (+ 11436us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 05:02:18.991126 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 05:02:18.991172 (+ 46us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 05:02:18.992488 (+ 1316us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 05:02:18.992504 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 05:02:18.994249 (+ 1745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 05:02:18.994255 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 05:02:18.996843 (+ 2588us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 05:02:18.996852 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 05:02:18.999160 (+ 2308us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 05:02:18.999174 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 05:02:18.999533 (+ 359us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:02:18.999594 (+ 61us) client_negotiation.cc:769] Sending connection context
0806 05:02:18.999720 (+ 126us) client_negotiation.cc:240] Negotiation successful
0806 05:02:19.001090 (+ 1370us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":592,"mutex_wait_us":1583,"spinlock_wait_cycles":768,"thread_start_us":508,"threads_started":1}
I20260806 05:02:19.003291 4578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:02:18.979137 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48756 (local address 127.9.96.67:33743)
0806 05:02:18.990293 (+ 11156us) server_negotiation.cc:206] Beginning negotiation
0806 05:02:18.990304 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 05:02:18.990342 (+ 38us) server_negotiation.cc:407] Connection header received
0806 05:02:18.990449 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 05:02:18.990458 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 05:02:18.990730 (+ 272us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 05:02:18.991759 (+ 1029us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 05:02:18.992817 (+ 1058us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:02:18.994359 (+ 1542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:02:18.997113 (+ 2754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:02:18.998669 (+ 1556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:02:18.998786 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:02:18.999044 (+ 258us) server_negotiation.cc:1035] Waiting for connection context
0806 05:02:19.002804 (+ 3760us) server_negotiation.cc:299] Negotiation successful
0806 05:02:19.003016 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10955,"thread_start_us":10804,"threads_started":1}
I20260806 05:02:19.005239 4580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:02:18.980381 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.65:36897 (local address 127.0.0.1:51258)
0806 05:02:18.992668 (+ 12287us) negotiation.cc:107] Waiting for socket to connect
0806 05:02:18.992688 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0806 05:02:18.992801 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 05:02:18.997911 (+ 5110us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 05:02:18.997916 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 05:02:18.997940 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 05:02:18.998256 (+ 316us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 05:02:18.998267 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 05:02:19.000726 (+ 2459us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 05:02:19.000731 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 05:02:19.002420 (+ 1689us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 05:02:19.002433 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 05:02:19.004376 (+ 1943us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 05:02:19.004400 (+ 24us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 05:02:19.004586 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:02:19.004621 (+ 35us) client_negotiation.cc:769] Sending connection context
0806 05:02:19.004729 (+ 108us) client_negotiation.cc:240] Negotiation successful
0806 05:02:19.004868 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12141,"mutex_wait_us":188,"thread_start_us":11910,"threads_started":1}
I20260806 05:02:19.005333 4583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:02:18.980903 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51258 (local address 127.9.96.65:36897)
0806 05:02:18.991999 (+ 11096us) server_negotiation.cc:206] Beginning negotiation
0806 05:02:18.992009 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 05:02:18.992934 (+ 925us) server_negotiation.cc:407] Connection header received
0806 05:02:18.993044 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 05:02:18.993052 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 05:02:18.997404 (+ 4352us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 05:02:18.997656 (+ 252us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 05:02:18.998445 (+ 789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:02:19.000080 (+ 1635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:02:19.002640 (+ 2560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:02:19.004158 (+ 1518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:02:19.004306 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:02:19.004597 (+ 291us) server_negotiation.cc:1035] Waiting for connection context
0806 05:02:19.004795 (+ 198us) server_negotiation.cc:299] Negotiation successful
0806 05:02:19.004973 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10928,"thread_start_us":10790,"threads_started":1}
I20260806 05:02:19.006624 4277 tablet_service.cc:1467] Processing CreateTablet for tablet d4428eb8e0c34e609ac2d4679c505f10 (DEFAULT_TABLE table=test-table [id=988be776f453453296a492686615a231]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 05:02:19.006624 4457 tablet_service.cc:1467] Processing CreateTablet for tablet d4428eb8e0c34e609ac2d4679c505f10 (DEFAULT_TABLE table=test-table [id=988be776f453453296a492686615a231]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 05:02:19.007839 4457 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d4428eb8e0c34e609ac2d4679c505f10. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 05:02:19.007931 4277 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d4428eb8e0c34e609ac2d4679c505f10. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 05:02:19.010926 4087 tablet_service.cc:1467] Processing CreateTablet for tablet d4428eb8e0c34e609ac2d4679c505f10 (DEFAULT_TABLE table=test-table [id=988be776f453453296a492686615a231]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 05:02:19.012315 4087 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d4428eb8e0c34e609ac2d4679c505f10. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 05:02:19.019042 4178 heartbeater.cc:503] Master 127.9.96.126:41613 was elected leader, sending a full tablet report...
I20260806 05:02:19.019477 4584 tablet_bootstrap.cc:492] T d4428eb8e0c34e609ac2d4679c505f10 P 593860df0812495eb4254fcd36b0a511: Bootstrap starting.
I20260806 05:02:19.023052 4584 tablet_bootstrap.cc:654] T d4428eb8e0c34e609ac2d4679c505f10 P 593860df0812495eb4254fcd36b0a511: Neither blocks nor log segments found. Creating new log.
I20260806 05:02:19.024150 4584 log.cc:824] T d4428eb8e0c34e609ac2d4679c505f10 P 593860df0812495eb4254fcd36b0a511: Log is configured to *not* fsync() on all Append() calls
I20260806 05:02:19.027740 4588 tablet_bootstrap.cc:492] T d4428eb8e0c34e609ac2d4679c505f10 P b2e228824a4a4af9a0aa32913c0cbf2a: Bootstrap starting.
I20260806 05:02:19.028290 4585 tablet_bootstrap.cc:492] T d4428eb8e0c34e609ac2d4679c505f10 P 1579aec8cc7f46a8a4260aa3e4e942e3: Bootstrap starting.
I20260806 05:02:19.028739 4584 tablet_bootstrap.cc:492] T d4428eb8e0c34e609ac2d4679c505f10 P 593860df0812495eb4254fcd36b0a511: No bootstrap required, opened a new log
I20260806 05:02:19.029027 4584 ts_tablet_manager.cc:1397] T d4428eb8e0c34e609ac2d4679c505f10 P 593860df0812495eb4254fcd36b0a511: Time spent bootstrapping tablet: real 0.010s user 0.005s sys 0.004s
I20260806 05:02:19.031137 4585 tablet_bootstrap.cc:654] T d4428eb8e0c34e609ac2d4679c505f10 P 1579aec8cc7f46a8a4260aa3e4e942e3: Neither blocks nor log segments found. Creating new log.
I20260806 05:02:19.031244 4588 tablet_bootstrap.cc:654] T d4428eb8e0c34e609ac2d4679c505f10 P b2e228824a4a4af9a0aa32913c0cbf2a: Neither blocks nor log segments found. Creating new log.
I20260806 05:02:19.032171 4585 log.cc:824] T d4428eb8e0c34e609ac2d4679c505f10 P 1579aec8cc7f46a8a4260aa3e4e942e3: Log is configured to *not* fsync() on all Append() calls
I20260806 05:02:19.032455 4588 log.cc:824] T d4428eb8e0c34e609ac2d4679c505f10 P b2e228824a4a4af9a0aa32913c0cbf2a: Log is configured to *not* fsync() on all Append() calls
I20260806 05:02:19.035308 4585 tablet_bootstrap.cc:492] T d4428eb8e0c34e609ac2d4679c505f10 P 1579aec8cc7f46a8a4260aa3e4e942e3: No bootstrap required, opened a new log
I20260806 05:02:19.035307 4588 tablet_bootstrap.cc:492] T d4428eb8e0c34e609ac2d4679c505f10 P b2e228824a4a4af9a0aa32913c0cbf2a: No bootstrap required, opened a new log
I20260806 05:02:19.035593 4585 ts_tablet_manager.cc:1397] T d4428eb8e0c34e609ac2d4679c505f10 P 1579aec8cc7f46a8a4260aa3e4e942e3: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260806 05:02:19.035593 4588 ts_tablet_manager.cc:1397] T d4428eb8e0c34e609ac2d4679c505f10 P b2e228824a4a4af9a0aa32913c0cbf2a: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.006s
I20260806 05:02:19.037760 4584 raft_consensus.cc:348] T d4428eb8e0c34e609ac2d4679c505f10 P 593860df0812495eb4254fcd36b0a511 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "593860df0812495eb4254fcd36b0a511" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 43299 } } peers { permanent_uuid: "1579aec8cc7f46a8a4260aa3e4e942e3" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 33743 } } peers { permanent_uuid: "b2e228824a4a4af9a0aa32913c0cbf2a" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 36897 } }
I20260806 05:02:19.038116 4584 raft_consensus.cc:374] T d4428eb8e0c34e609ac2d4679c505f10 P 593860df0812495eb4254fcd36b0a511 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 05:02:19.038225 4584 raft_consensus.cc:729] T d4428eb8e0c34e609ac2d4679c505f10 P 593860df0812495eb4254fcd36b0a511 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 593860df0812495eb4254fcd36b0a511, State: Initialized, Role: FOLLOWER
I20260806 05:02:19.038932 4584 consensus_queue.cc:260] T d4428eb8e0c34e609ac2d4679c505f10 P 593860df0812495eb4254fcd36b0a511 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "593860df0812495eb4254fcd36b0a511" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 43299 } } peers { permanent_uuid: "1579aec8cc7f46a8a4260aa3e4e942e3" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 33743 } } peers { permanent_uuid: "b2e228824a4a4af9a0aa32913c0cbf2a" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 36897 } }
I20260806 05:02:19.044859 4584 ts_tablet_manager.cc:1428] T d4428eb8e0c34e609ac2d4679c505f10 P 593860df0812495eb4254fcd36b0a511: Time spent starting tablet: real 0.016s user 0.010s sys 0.005s
I20260806 05:02:19.045289 4366 heartbeater.cc:503] Master 127.9.96.126:41613 was elected leader, sending a full tablet report...
I20260806 05:02:19.046355 4588 raft_consensus.cc:348] T d4428eb8e0c34e609ac2d4679c505f10 P b2e228824a4a4af9a0aa32913c0cbf2a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "593860df0812495eb4254fcd36b0a511" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 43299 } } peers { permanent_uuid: "1579aec8cc7f46a8a4260aa3e4e942e3" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 33743 } } peers { permanent_uuid: "b2e228824a4a4af9a0aa32913c0cbf2a" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 36897 } }
I20260806 05:02:19.046351 4585 raft_consensus.cc:348] T d4428eb8e0c34e609ac2d4679c505f10 P 1579aec8cc7f46a8a4260aa3e4e942e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "593860df0812495eb4254fcd36b0a511" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 43299 } } peers { permanent_uuid: "1579aec8cc7f46a8a4260aa3e4e942e3" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 33743 } } peers { permanent_uuid: "b2e228824a4a4af9a0aa32913c0cbf2a" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 36897 } }
I20260806 05:02:19.046809 4588 raft_consensus.cc:374] T d4428eb8e0c34e609ac2d4679c505f10 P b2e228824a4a4af9a0aa32913c0cbf2a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 05:02:19.046824 4585 raft_consensus.cc:374] T d4428eb8e0c34e609ac2d4679c505f10 P 1579aec8cc7f46a8a4260aa3e4e942e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 05:02:19.046928 4588 raft_consensus.cc:729] T d4428eb8e0c34e609ac2d4679c505f10 P b2e228824a4a4af9a0aa32913c0cbf2a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b2e228824a4a4af9a0aa32913c0cbf2a, State: Initialized, Role: FOLLOWER
I20260806 05:02:19.046963 4585 raft_consensus.cc:729] T d4428eb8e0c34e609ac2d4679c505f10 P 1579aec8cc7f46a8a4260aa3e4e942e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1579aec8cc7f46a8a4260aa3e4e942e3, State: Initialized, Role: FOLLOWER
I20260806 05:02:19.047571 4585 consensus_queue.cc:260] T d4428eb8e0c34e609ac2d4679c505f10 P 1579aec8cc7f46a8a4260aa3e4e942e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "593860df0812495eb4254fcd36b0a511" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 43299 } } peers { permanent_uuid: "1579aec8cc7f46a8a4260aa3e4e942e3" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 33743 } } peers { permanent_uuid: "b2e228824a4a4af9a0aa32913c0cbf2a" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 36897 } }
I20260806 05:02:19.047571 4588 consensus_queue.cc:260] T d4428eb8e0c34e609ac2d4679c505f10 P b2e228824a4a4af9a0aa32913c0cbf2a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "593860df0812495eb4254fcd36b0a511" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 43299 } } peers { permanent_uuid: "1579aec8cc7f46a8a4260aa3e4e942e3" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 33743 } } peers { permanent_uuid: "b2e228824a4a4af9a0aa32913c0cbf2a" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 36897 } }
I20260806 05:02:19.050084 4561 heartbeater.cc:503] Master 127.9.96.126:41613 was elected leader, sending a full tablet report...
I20260806 05:02:19.050405 4588 ts_tablet_manager.cc:1428] T d4428eb8e0c34e609ac2d4679c505f10 P b2e228824a4a4af9a0aa32913c0cbf2a: Time spent starting tablet: real 0.015s user 0.013s sys 0.001s
I20260806 05:02:19.050671 4585 ts_tablet_manager.cc:1428] T d4428eb8e0c34e609ac2d4679c505f10 P 1579aec8cc7f46a8a4260aa3e4e942e3: Time spent starting tablet: real 0.015s user 0.014s sys 0.000s
I20260806 05:02:19.158749 4594 raft_consensus.cc:482] T d4428eb8e0c34e609ac2d4679c505f10 P 1579aec8cc7f46a8a4260aa3e4e942e3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 05:02:19.159157 4594 raft_consensus.cc:504] T d4428eb8e0c34e609ac2d4679c505f10 P 1579aec8cc7f46a8a4260aa3e4e942e3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "593860df0812495eb4254fcd36b0a511" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 43299 } } peers { permanent_uuid: "1579aec8cc7f46a8a4260aa3e4e942e3" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 33743 } } peers { permanent_uuid: "b2e228824a4a4af9a0aa32913c0cbf2a" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 36897 } }
I20260806 05:02:19.161528 4594 leader_election.cc:288] T d4428eb8e0c34e609ac2d4679c505f10 P 1579aec8cc7f46a8a4260aa3e4e942e3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 593860df0812495eb4254fcd36b0a511 (127.9.96.66:43299), b2e228824a4a4af9a0aa32913c0cbf2a (127.9.96.65:36897)
I20260806 05:02:19.168138 4583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:02:19.162050 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.96.67:35823 (local address 127.9.96.65:36897)
0806 05:02:19.162316 (+ 266us) server_negotiation.cc:206] Beginning negotiation
0806 05:02:19.162326 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 05:02:19.162447 (+ 121us) server_negotiation.cc:407] Connection header received
0806 05:02:19.162697 (+ 250us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 05:02:19.162705 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 05:02:19.162801 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 05:02:19.162908 (+ 107us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 05:02:19.163434 (+ 526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:02:19.164545 (+ 1111us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:02:19.166346 (+ 1801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:02:19.167210 (+ 864us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:02:19.167390 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:02:19.167494 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0806 05:02:19.167751 (+ 257us) server_negotiation.cc:299] Negotiation successful
0806 05:02:19.167877 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":107}
I20260806 05:02:19.168156 4563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:02:19.161988 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.65:36897 (local address 127.9.96.67:35823)
0806 05:02:19.162341 (+ 353us) negotiation.cc:107] Waiting for socket to connect
0806 05:02:19.162361 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0806 05:02:19.162509 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 05:02:19.162936 (+ 427us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 05:02:19.162941 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 05:02:19.162968 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 05:02:19.163200 (+ 232us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 05:02:19.163208 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 05:02:19.164717 (+ 1509us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 05:02:19.164721 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 05:02:19.166133 (+ 1412us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 05:02:19.166146 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 05:02:19.167392 (+ 1246us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 05:02:19.167398 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 05:02:19.167569 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:02:19.167593 (+ 24us) client_negotiation.cc:769] Sending connection context
0806 05:02:19.167703 (+ 110us) client_negotiation.cc:240] Negotiation successful
0806 05:02:19.167838 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":231}
W20260806 05:02:19.169013 4562 tablet.cc:2369] T d4428eb8e0c34e609ac2d4679c505f10 P 1579aec8cc7f46a8a4260aa3e4e942e3: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 05:02:19.169169 4595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:02:19.162047 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.66:43299 (local address 127.9.96.67:37815)
0806 05:02:19.162948 (+ 901us) negotiation.cc:107] Waiting for socket to connect
0806 05:02:19.162965 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 05:02:19.163165 (+ 200us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 05:02:19.163596 (+ 431us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 05:02:19.163599 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 05:02:19.163618 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 05:02:19.163902 (+ 284us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 05:02:19.163909 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 05:02:19.165180 (+ 1271us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 05:02:19.165186 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 05:02:19.166945 (+ 1759us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 05:02:19.166965 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 05:02:19.168237 (+ 1272us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 05:02:19.168244 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 05:02:19.168472 (+ 228us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:02:19.168512 (+ 40us) client_negotiation.cc:769] Sending connection context
0806 05:02:19.168640 (+ 128us) client_negotiation.cc:240] Negotiation successful
0806 05:02:19.168790 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":775,"thread_start_us":504,"threads_started":1}
I20260806 05:02:19.169153 4577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:02:19.162196 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.96.67:37815 (local address 127.9.96.66:43299)
0806 05:02:19.162381 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0806 05:02:19.162389 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 05:02:19.163083 (+ 694us) server_negotiation.cc:407] Connection header received
0806 05:02:19.163330 (+ 247us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 05:02:19.163339 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 05:02:19.163470 (+ 131us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 05:02:19.163588 (+ 118us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 05:02:19.164095 (+ 507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:02:19.165004 (+ 909us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:02:19.167148 (+ 2144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:02:19.168044 (+ 896us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:02:19.168205 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:02:19.168310 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0806 05:02:19.168687 (+ 377us) server_negotiation.cc:299] Negotiation successful
0806 05:02:19.168826 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260806 05:02:19.169323 4120 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d4428eb8e0c34e609ac2d4679c505f10" candidate_uuid: "1579aec8cc7f46a8a4260aa3e4e942e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2e228824a4a4af9a0aa32913c0cbf2a" is_pre_election: true
I20260806 05:02:19.169929 4120 raft_consensus.cc:2454] T d4428eb8e0c34e609ac2d4679c505f10 P b2e228824a4a4af9a0aa32913c0cbf2a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1579aec8cc7f46a8a4260aa3e4e942e3 in term 0.
I20260806 05:02:19.170436 4305 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d4428eb8e0c34e609ac2d4679c505f10" candidate_uuid: "1579aec8cc7f46a8a4260aa3e4e942e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "593860df0812495eb4254fcd36b0a511" is_pre_election: true
I20260806 05:02:19.170617 4393 leader_election.cc:302] T d4428eb8e0c34e609ac2d4679c505f10 P 1579aec8cc7f46a8a4260aa3e4e942e3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1579aec8cc7f46a8a4260aa3e4e942e3, b2e228824a4a4af9a0aa32913c0cbf2a; no voters:
I20260806 05:02:19.171029 4594 raft_consensus.cc:2781] T d4428eb8e0c34e609ac2d4679c505f10 P 1579aec8cc7f46a8a4260aa3e4e942e3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 05:02:19.171029 4305 raft_consensus.cc:2454] T d4428eb8e0c34e609ac2d4679c505f10 P 593860df0812495eb4254fcd36b0a511 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1579aec8cc7f46a8a4260aa3e4e942e3 in term 0.
I20260806 05:02:19.171221 4594 raft_consensus.cc:482] T d4428eb8e0c34e609ac2d4679c505f10 P 1579aec8cc7f46a8a4260aa3e4e942e3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 05:02:19.171392 4594 raft_consensus.cc:3037] T d4428eb8e0c34e609ac2d4679c505f10 P 1579aec8cc7f46a8a4260aa3e4e942e3 [term 0 FOLLOWER]: Advancing to term 1
I20260806 05:02:19.173751 4594 raft_consensus.cc:504] T d4428eb8e0c34e609ac2d4679c505f10 P 1579aec8cc7f46a8a4260aa3e4e942e3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "593860df0812495eb4254fcd36b0a511" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 43299 } } peers { permanent_uuid: "1579aec8cc7f46a8a4260aa3e4e942e3" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 33743 } } peers { permanent_uuid: "b2e228824a4a4af9a0aa32913c0cbf2a" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 36897 } }
I20260806 05:02:19.174410 4594 leader_election.cc:288] T d4428eb8e0c34e609ac2d4679c505f10 P 1579aec8cc7f46a8a4260aa3e4e942e3 [CANDIDATE]: Term 1 election: Requested vote from peers 593860df0812495eb4254fcd36b0a511 (127.9.96.66:43299), b2e228824a4a4af9a0aa32913c0cbf2a (127.9.96.65:36897)
I20260806 05:02:19.174976 4305 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d4428eb8e0c34e609ac2d4679c505f10" candidate_uuid: "1579aec8cc7f46a8a4260aa3e4e942e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "593860df0812495eb4254fcd36b0a511"
I20260806 05:02:19.174976 4120 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d4428eb8e0c34e609ac2d4679c505f10" candidate_uuid: "1579aec8cc7f46a8a4260aa3e4e942e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2e228824a4a4af9a0aa32913c0cbf2a"
I20260806 05:02:19.175220 4305 raft_consensus.cc:3037] T d4428eb8e0c34e609ac2d4679c505f10 P 593860df0812495eb4254fcd36b0a511 [term 0 FOLLOWER]: Advancing to term 1
I20260806 05:02:19.175221 4120 raft_consensus.cc:3037] T d4428eb8e0c34e609ac2d4679c505f10 P b2e228824a4a4af9a0aa32913c0cbf2a [term 0 FOLLOWER]: Advancing to term 1
I20260806 05:02:19.178720 4120 raft_consensus.cc:2454] T d4428eb8e0c34e609ac2d4679c505f10 P b2e228824a4a4af9a0aa32913c0cbf2a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1579aec8cc7f46a8a4260aa3e4e942e3 in term 1.
I20260806 05:02:19.178740 4305 raft_consensus.cc:2454] T d4428eb8e0c34e609ac2d4679c505f10 P 593860df0812495eb4254fcd36b0a511 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1579aec8cc7f46a8a4260aa3e4e942e3 in term 1.
I20260806 05:02:19.179188 4393 leader_election.cc:302] T d4428eb8e0c34e609ac2d4679c505f10 P 1579aec8cc7f46a8a4260aa3e4e942e3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1579aec8cc7f46a8a4260aa3e4e942e3, b2e228824a4a4af9a0aa32913c0cbf2a; no voters:
I20260806 05:02:19.179524 4594 raft_consensus.cc:2781] T d4428eb8e0c34e609ac2d4679c505f10 P 1579aec8cc7f46a8a4260aa3e4e942e3 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 05:02:19.180025 4594 raft_consensus.cc:686] T d4428eb8e0c34e609ac2d4679c505f10 P 1579aec8cc7f46a8a4260aa3e4e942e3 [term 1 LEADER]: Becoming Leader. State: Replica: 1579aec8cc7f46a8a4260aa3e4e942e3, State: Running, Role: LEADER
I20260806 05:02:19.180586 4594 consensus_queue.cc:237] T d4428eb8e0c34e609ac2d4679c505f10 P 1579aec8cc7f46a8a4260aa3e4e942e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "593860df0812495eb4254fcd36b0a511" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 43299 } } peers { permanent_uuid: "1579aec8cc7f46a8a4260aa3e4e942e3" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 33743 } } peers { permanent_uuid: "b2e228824a4a4af9a0aa32913c0cbf2a" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 36897 } }
I20260806 05:02:19.185325 3908 catalog_manager.cc:5495] T d4428eb8e0c34e609ac2d4679c505f10 P 1579aec8cc7f46a8a4260aa3e4e942e3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1579aec8cc7f46a8a4260aa3e4e942e3 (127.9.96.67). New cstate: current_term: 1 leader_uuid: "1579aec8cc7f46a8a4260aa3e4e942e3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "593860df0812495eb4254fcd36b0a511" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 43299 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1579aec8cc7f46a8a4260aa3e4e942e3" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 33743 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b2e228824a4a4af9a0aa32913c0cbf2a" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 36897 } health_report { overall_health: UNKNOWN } } }
W20260806 05:02:19.205492 4367 tablet.cc:2369] T d4428eb8e0c34e609ac2d4679c505f10 P 593860df0812495eb4254fcd36b0a511: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 05:02:19.243448 4578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:02:19.238191 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48766 (local address 127.9.96.67:33743)
0806 05:02:19.238384 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0806 05:02:19.238388 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 05:02:19.238404 (+ 16us) server_negotiation.cc:407] Connection header received
0806 05:02:19.238551 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 05:02:19.238556 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 05:02:19.238666 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 05:02:19.238819 (+ 153us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 05:02:19.239258 (+ 439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:02:19.240077 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:02:19.241476 (+ 1399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:02:19.241873 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:02:19.241971 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:02:19.242436 (+ 465us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 05:02:19.242945 (+ 509us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 05:02:19.243064 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0806 05:02:19.243143 (+ 79us) server_negotiation.cc:299] Negotiation successful
0806 05:02:19.243207 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
W20260806 05:02:19.247696 4179 tablet.cc:2369] T d4428eb8e0c34e609ac2d4679c505f10 P b2e228824a4a4af9a0aa32913c0cbf2a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 05:02:19.251565 4120 raft_consensus.cc:1261] T d4428eb8e0c34e609ac2d4679c505f10 P b2e228824a4a4af9a0aa32913c0cbf2a [term 1 FOLLOWER]: Refusing update from remote peer 1579aec8cc7f46a8a4260aa3e4e942e3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 05:02:19.251751 4305 raft_consensus.cc:1261] T d4428eb8e0c34e609ac2d4679c505f10 P 593860df0812495eb4254fcd36b0a511 [term 1 FOLLOWER]: Refusing update from remote peer 1579aec8cc7f46a8a4260aa3e4e942e3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 05:02:19.252614 4596 consensus_queue.cc:1035] T d4428eb8e0c34e609ac2d4679c505f10 P 1579aec8cc7f46a8a4260aa3e4e942e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b2e228824a4a4af9a0aa32913c0cbf2a" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 36897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 05:02:19.252942 4594 consensus_queue.cc:1035] T d4428eb8e0c34e609ac2d4679c505f10 P 1579aec8cc7f46a8a4260aa3e4e942e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "593860df0812495eb4254fcd36b0a511" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 43299 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 05:02:19.267546 4602 mvcc.cc:205] Tried to move back new op lower bound from 7315425440764788736 to 7315425440487555072. Current Snapshot: MvccSnapshot[applied={T|T < 7315425440764788736}]
I20260806 05:02:19.281805 3907 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47998:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 05:02:19.282124 3907 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47998:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 05:02:19.286384 3907 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 65d2bdeb1989438991a5c4a5c90d8089: Sending DeleteTablet for 3 replicas of tablet d4428eb8e0c34e609ac2d4679c505f10
I20260806 05:02:19.287732 4277 tablet_service.cc:1514] Processing DeleteTablet for tablet d4428eb8e0c34e609ac2d4679c505f10 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 05:02:19 UTC) from {username='slave'} at 127.0.0.1:57548
I20260806 05:02:19.287839 4087 tablet_service.cc:1514] Processing DeleteTablet for tablet d4428eb8e0c34e609ac2d4679c505f10 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 05:02:19 UTC) from {username='slave'} at 127.0.0.1:51258
I20260806 05:02:19.288585 4457 tablet_service.cc:1514] Processing DeleteTablet for tablet d4428eb8e0c34e609ac2d4679c505f10 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 05:02:19 UTC) from {username='slave'} at 127.0.0.1:48756
I20260806 05:02:19.289005 4616 tablet_replica.cc:330] T d4428eb8e0c34e609ac2d4679c505f10 P 593860df0812495eb4254fcd36b0a511: stopping tablet replica
I20260806 05:02:19.289501 4616 raft_consensus.cc:2229] T d4428eb8e0c34e609ac2d4679c505f10 P 593860df0812495eb4254fcd36b0a511 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 05:02:19.289798 9601 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu pid 4000
I20260806 05:02:19.290164 4616 raft_consensus.cc:2258] T d4428eb8e0c34e609ac2d4679c505f10 P 593860df0812495eb4254fcd36b0a511 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 05:02:19.291018 4614 tablet_replica.cc:330] T d4428eb8e0c34e609ac2d4679c505f10 P b2e228824a4a4af9a0aa32913c0cbf2a: stopping tablet replica
I20260806 05:02:19.291556 4614 raft_consensus.cc:2229] T d4428eb8e0c34e609ac2d4679c505f10 P b2e228824a4a4af9a0aa32913c0cbf2a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 05:02:19.292104 4614 raft_consensus.cc:2258] T d4428eb8e0c34e609ac2d4679c505f10 P b2e228824a4a4af9a0aa32913c0cbf2a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 05:02:19.295149 4616 ts_tablet_manager.cc:1905] T d4428eb8e0c34e609ac2d4679c505f10 P 593860df0812495eb4254fcd36b0a511: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 05:02:19.299993 4614 ts_tablet_manager.cc:1905] T d4428eb8e0c34e609ac2d4679c505f10 P b2e228824a4a4af9a0aa32913c0cbf2a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 05:02:19.300932 4617 tablet_replica.cc:330] T d4428eb8e0c34e609ac2d4679c505f10 P 1579aec8cc7f46a8a4260aa3e4e942e3: stopping tablet replica
I20260806 05:02:19.301430 4617 raft_consensus.cc:2229] T d4428eb8e0c34e609ac2d4679c505f10 P 1579aec8cc7f46a8a4260aa3e4e942e3 [term 1 LEADER]: Raft consensus shutting down.
I20260806 05:02:19.302182 4617 raft_consensus.cc:2258] T d4428eb8e0c34e609ac2d4679c505f10 P 1579aec8cc7f46a8a4260aa3e4e942e3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 05:02:19.303900 4616 ts_tablet_manager.cc:1918] T d4428eb8e0c34e609ac2d4679c505f10 P 593860df0812495eb4254fcd36b0a511: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 05:02:19.304131 4616 log.cc:1196] T d4428eb8e0c34e609ac2d4679c505f10 P 593860df0812495eb4254fcd36b0a511: Deleting WAL directory at /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785992227050689-9601-0/minicluster-data/ts-1/wal/wals/d4428eb8e0c34e609ac2d4679c505f10
I20260806 05:02:19.304471 4617 ts_tablet_manager.cc:1905] T d4428eb8e0c34e609ac2d4679c505f10 P 1579aec8cc7f46a8a4260aa3e4e942e3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 05:02:19.304910 4616 ts_tablet_manager.cc:1939] T d4428eb8e0c34e609ac2d4679c505f10 P 593860df0812495eb4254fcd36b0a511: Deleting consensus metadata
I20260806 05:02:19.306929 3888 catalog_manager.cc:4841] TS 593860df0812495eb4254fcd36b0a511 (127.9.96.66:43299): tablet d4428eb8e0c34e609ac2d4679c505f10 (table test-table [id=988be776f453453296a492686615a231]) successfully deleted
I20260806 05:02:19.309425 4583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:02:19.292902 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51266 (local address 127.9.96.65:36897)
0806 05:02:19.293130 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0806 05:02:19.293139 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 05:02:19.300687 (+ 7548us) server_negotiation.cc:407] Connection header received
0806 05:02:19.300911 (+ 224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 05:02:19.300918 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 05:02:19.301034 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 05:02:19.301154 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0806 05:02:19.301961 (+ 807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:02:19.303096 (+ 1135us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:02:19.305211 (+ 2115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:02:19.305861 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:02:19.305967 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:02:19.307564 (+ 1597us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 05:02:19.307574 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 05:02:19.307591 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 05:02:19.308399 (+ 808us) server_negotiation.cc:922] calling sasl_server_start()
0806 05:02:19.308461 (+ 62us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 05:02:19.308485 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 05:02:19.308584 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0806 05:02:19.308867 (+ 283us) server_negotiation.cc:299] Negotiation successful
0806 05:02:19.308996 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260806 05:02:19.310640 4171 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 05:02:19.315071 4617 ts_tablet_manager.cc:1918] T d4428eb8e0c34e609ac2d4679c505f10 P 1579aec8cc7f46a8a4260aa3e4e942e3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 05:02:19.315290 4617 log.cc:1196] T d4428eb8e0c34e609ac2d4679c505f10 P 1579aec8cc7f46a8a4260aa3e4e942e3: Deleting WAL directory at /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785992227050689-9601-0/minicluster-data/ts-2/wal/wals/d4428eb8e0c34e609ac2d4679c505f10
I20260806 05:02:19.316103 4617 ts_tablet_manager.cc:1939] T d4428eb8e0c34e609ac2d4679c505f10 P 1579aec8cc7f46a8a4260aa3e4e942e3: Deleting consensus metadata
I20260806 05:02:19.318380 3889 catalog_manager.cc:4841] TS 1579aec8cc7f46a8a4260aa3e4e942e3 (127.9.96.67:33743): tablet d4428eb8e0c34e609ac2d4679c505f10 (table test-table [id=988be776f453453296a492686615a231]) successfully deleted
I20260806 05:02:19.405028 9601 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu with pid 4000
I20260806 05:02:19.432940 9601 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu pid 4183
W20260806 05:02:19.432986 3886 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.9.96.65:36897 (error 108)
I20260806 05:02:19.435056 4580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:02:19.434550 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.65:36897 (local address 127.0.0.1:51278)
0806 05:02:19.434751 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0806 05:02:19.434881 (+ 130us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.96.65:36897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":76}
W20260806 05:02:19.435562 3886 catalog_manager.cc:4584] TS b2e228824a4a4af9a0aa32913c0cbf2a (127.9.96.65:36897): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet d4428eb8e0c34e609ac2d4679c505f10: Network error: Client connection negotiation failed: client connection to 127.9.96.65:36897: connect: Connection refused (error 111)
I20260806 05:02:19.443275 4577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:02:19.433870 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57562 (local address 127.9.96.66:43299)
0806 05:02:19.434062 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0806 05:02:19.434068 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 05:02:19.435466 (+ 1398us) server_negotiation.cc:407] Connection header received
0806 05:02:19.435546 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 05:02:19.435553 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 05:02:19.435665 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 05:02:19.435764 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0806 05:02:19.436425 (+ 661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:02:19.437561 (+ 1136us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:02:19.439560 (+ 1999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:02:19.440203 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:02:19.440350 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:02:19.441598 (+ 1248us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 05:02:19.441609 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 05:02:19.441629 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 05:02:19.442430 (+ 801us) server_negotiation.cc:922] calling sasl_server_start()
0806 05:02:19.442493 (+ 63us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 05:02:19.442519 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 05:02:19.442636 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0806 05:02:19.442805 (+ 169us) server_negotiation.cc:299] Negotiation successful
0806 05:02:19.442923 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260806 05:02:19.444177 4360 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 05:02:19.489529 4580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:02:19.488874 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.65:36897 (local address 127.0.0.1:51290)
0806 05:02:19.489207 (+ 333us) negotiation.cc:107] Waiting for socket to connect
0806 05:02:19.489333 (+ 126us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.96.65:36897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":182}
I20260806 05:02:19.490984 4580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:02:19.490572 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.65:36897 (local address 127.0.0.1:51292)
0806 05:02:19.490761 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0806 05:02:19.490849 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.96.65:36897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100}
I20260806 05:02:19.551653 4580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:02:19.551003 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.65:36897 (local address 127.0.0.1:51304)
0806 05:02:19.551337 (+ 334us) negotiation.cc:107] Waiting for socket to connect
0806 05:02:19.551476 (+ 139us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.96.65:36897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":173}
I20260806 05:02:19.553057 4580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:02:19.552559 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.65:36897 (local address 127.0.0.1:51320)
0806 05:02:19.552788 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0806 05:02:19.552890 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.96.65:36897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115}
I20260806 05:02:19.554826 9601 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu with pid 4183
I20260806 05:02:19.580744 9601 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu pid 4370
I20260806 05:02:19.589040 4578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:02:19.581719 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48782 (local address 127.9.96.67:33743)
0806 05:02:19.581938 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0806 05:02:19.581944 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 05:02:19.581968 (+ 24us) server_negotiation.cc:407] Connection header received
0806 05:02:19.582050 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 05:02:19.582056 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 05:02:19.582154 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 05:02:19.582293 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0806 05:02:19.582671 (+ 378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:02:19.583895 (+ 1224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:02:19.585232 (+ 1337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:02:19.585913 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:02:19.586050 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:02:19.587131 (+ 1081us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 05:02:19.587142 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 05:02:19.587161 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 05:02:19.588101 (+ 940us) server_negotiation.cc:922] calling sasl_server_start()
0806 05:02:19.588162 (+ 61us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 05:02:19.588190 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 05:02:19.588319 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0806 05:02:19.588499 (+ 180us) server_negotiation.cc:299] Negotiation successful
0806 05:02:19.588615 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260806 05:02:19.589790 4555 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 05:02:19.633898 4580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:02:19.633319 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.65:36897 (local address 127.0.0.1:51334)
0806 05:02:19.633568 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0806 05:02:19.633689 (+ 121us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.96.65:36897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":95}
I20260806 05:02:19.635066 4580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:02:19.634593 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.65:36897 (local address 127.0.0.1:51346)
0806 05:02:19.634844 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0806 05:02:19.634929 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.96.65:36897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":141}
I20260806 05:02:19.710268 9601 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu with pid 4370
I20260806 05:02:19.734587 9601 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu pid 3864
I20260806 05:02:19.735816 3953 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 05:02:19.844081 4580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:02:19.843367 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.65:36897 (local address 127.0.0.1:51360)
0806 05:02:19.843784 (+ 417us) negotiation.cc:107] Waiting for socket to connect
0806 05:02:19.843905 (+ 121us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.96.65:36897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":221}
I20260806 05:02:19.844099 9601 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu with pid 3864
2026-08-06T05:02:19Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (2973 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785992227050689-9601-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 05:02:19 dist-test-slave-x4qh krb5kdc[4634](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 05:02:19 dist-test-slave-x4qh krb5kdc[4634](info): set up 2 sockets
Aug 06 05:02:19 dist-test-slave-x4qh krb5kdc[4634](info): commencing operation
krb5kdc: starting...
W20260806 05:02:21.199620 9601 mini_kdc.cc:121] Time spent starting KDC: real 1.305s user 0.002s sys 0.040s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785992227050689-9601-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785992227050689-9601-0/krb5kdc/test-user.keytab.
Aug 06 05:02:21 dist-test-slave-x4qh krb5kdc[4634](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992541, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T05:02:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T05:02:21Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.96.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.96.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.96.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.126.keytab.
Entry for principal kudu/127.9.96.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.126.keytab.
I20260806 05:02:21.501134 9601 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
/tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785992227050689-9601-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785992227050689-9601-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785992227050689-9601-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785992227050689-9601-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.96.126:41365
--webserver_interface=127.9.96.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.126.keytab
--principal=kudu/127.9.96.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.96.84:36485
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.96.126:41365
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785992227050689-9601-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785992227050689-9601-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785992227050689-9601-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 05:02:21.843078 4651 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 05:02:21.843478 4651 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 05:02:21.843647 4651 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 05:02:21.850816 4651 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 05:02:21.850963 4651 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 05:02:21.851076 4651 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 05:02:21.851184 4651 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 05:02:21.851276 4651 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 05:02:21.860899 4651 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.96.84:36485
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785992227050689-9601-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785992227050689-9601-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.96.126:41365
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.126.keytab
--principal=kudu/127.9.96.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.96.126:41365
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785992227050689-9601-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.96.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785992227050689-9601-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 03:56:12 UTC on ccycloud.quasar-guijbm.root.comops.site
build id 4569618
ASAN enabled
I20260806 05:02:21.862241 4651 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 05:02:21.863579 4651 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 05:02:21.875911 4661 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 05:02:21.875944 4658 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 05:02:21.875984 4657 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 05:02:21.876993 4651 server_base.cc:1034] running on GCE node
I20260806 05:02:21.877794 4651 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 05:02:21.879251 4651 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 05:02:21.880487 4651 hybrid_clock.cc:648] HybridClock initialized: now 1785992541880454 us; error 69 us; skew 500 ppm
I20260806 05:02:21.880870 4651 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 05:02:21 dist-test-slave-x4qh krb5kdc[4634](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992541, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.96.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 05:02:21.884227 4651 init.cc:377] Logged in from keytab as kudu/127.9.96.126@KRBTEST.COM (short username kudu)
I20260806 05:02:21.885878 4651 webserver.cc:460] Webserver started at http://127.9.96.126:45959/ using document root <none> and password file <none>
I20260806 05:02:21.886627 4651 fs_manager.cc:359] Metadata directory not provided
I20260806 05:02:21.886797 4651 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 05:02:21.887145 4651 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 05:02:21.890337 4651 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785992227050689-9601-0/minicluster-data/master-0/data/instance:
uuid: "76f2527f84ce4061a7e0161394484916"
format_stamp: "Formatted at 2026-08-06 05:02:21 on dist-test-slave-x4qh"
server_key: "01e8b702bcec2fb4444a4589ce4e12ef"
server_key_iv: "6af2d22f90e07733ba0f3944162ce521"
server_key_version: "encryptionkey@0"
I20260806 05:02:21.891201 4651 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785992227050689-9601-0/minicluster-data/master-0/wal/instance:
uuid: "76f2527f84ce4061a7e0161394484916"
format_stamp: "Formatted at 2026-08-06 05:02:21 on dist-test-slave-x4qh"
server_key: "01e8b702bcec2fb4444a4589ce4e12ef"
server_key_iv: "6af2d22f90e07733ba0f3944162ce521"
server_key_version: "encryptionkey@0"
I20260806 05:02:21.896591 4651 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.002s
I20260806 05:02:21.900223 4672 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 05:02:21.901741 4651 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 05:02:21.901947 4651 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785992227050689-9601-0/minicluster-data/master-0/data,/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785992227050689-9601-0/minicluster-data/master-0/wal
uuid: "76f2527f84ce4061a7e0161394484916"
format_stamp: "Formatted at 2026-08-06 05:02:21 on dist-test-slave-x4qh"
server_key: "01e8b702bcec2fb4444a4589ce4e12ef"
server_key_iv: "6af2d22f90e07733ba0f3944162ce521"
server_key_version: "encryptionkey@0"
I20260806 05:02:21.902146 4651 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785992227050689-9601-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785992227050689-9601-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785992227050689-9601-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 05:02:21.939399 4651 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 05:02:21.943552 4651 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 05:02:21.943856 4651 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 05:02:21.962605 4651 rpc_server.cc:307] RPC server started. Bound to: 127.9.96.126:41365
I20260806 05:02:21.962692 4747 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.96.126:41365 every 8 connection(s)
I20260806 05:02:21.963989 4651 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785992227050689-9601-0/minicluster-data/master-0/data/info.pb
I20260806 05:02:21.967774 4748 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 05:02:21.972133 9601 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu as pid 4651
I20260806 05:02:21.972352 9601 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785992227050689-9601-0/minicluster-data/master-0/wal/instance
I20260806 05:02:21.972970 9601 external_mini_cluster.cc:1428] Setting key 2bc29d2896c6059e6e606fa3e46438c5
I20260806 05:02:21.979238 4748 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 76f2527f84ce4061a7e0161394484916: Bootstrap starting.
I20260806 05:02:21.982856 4748 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 76f2527f84ce4061a7e0161394484916: Neither blocks nor log segments found. Creating new log.
I20260806 05:02:21.984076 4748 log.cc:824] T 00000000000000000000000000000000 P 76f2527f84ce4061a7e0161394484916: Log is configured to *not* fsync() on all Append() calls
I20260806 05:02:21.987630 4748 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 76f2527f84ce4061a7e0161394484916: No bootstrap required, opened a new log
Aug 06 05:02:21 dist-test-slave-x4qh krb5kdc[4634](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992541, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.96.126@KRBTEST.COM
I20260806 05:02:21.995865 4748 raft_consensus.cc:348] T 00000000000000000000000000000000 P 76f2527f84ce4061a7e0161394484916 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76f2527f84ce4061a7e0161394484916" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 41365 } }
I20260806 05:02:21.996351 4748 raft_consensus.cc:374] T 00000000000000000000000000000000 P 76f2527f84ce4061a7e0161394484916 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 05:02:21.996460 4748 raft_consensus.cc:729] T 00000000000000000000000000000000 P 76f2527f84ce4061a7e0161394484916 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 76f2527f84ce4061a7e0161394484916, State: Initialized, Role: FOLLOWER
I20260806 05:02:21.997045 4748 consensus_queue.cc:260] T 00000000000000000000000000000000 P 76f2527f84ce4061a7e0161394484916 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76f2527f84ce4061a7e0161394484916" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 41365 } }
I20260806 05:02:21.997298 4748 raft_consensus.cc:388] T 00000000000000000000000000000000 P 76f2527f84ce4061a7e0161394484916 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 05:02:21.997449 4748 raft_consensus.cc:482] T 00000000000000000000000000000000 P 76f2527f84ce4061a7e0161394484916 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 05:02:21.997649 4748 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 76f2527f84ce4061a7e0161394484916 [term 0 FOLLOWER]: Advancing to term 1
I20260806 05:02:21.997704 4751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:02:21.974893 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42540 (local address 127.9.96.126:41365)
0806 05:02:21.975689 (+ 796us) server_negotiation.cc:206] Beginning negotiation
0806 05:02:21.975700 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 05:02:21.975747 (+ 47us) server_negotiation.cc:407] Connection header received
0806 05:02:21.977815 (+ 2068us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 05:02:21.977837 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 05:02:21.978180 (+ 343us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 05:02:21.978471 (+ 291us) server_negotiation.cc:226] Negotiated authn=SASL
0806 05:02:21.980377 (+ 1906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:02:21.981685 (+ 1308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:02:21.983592 (+ 1907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:02:21.984357 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:02:21.984514 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:02:21.989668 (+ 5154us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 05:02:21.989699 (+ 31us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 05:02:21.989713 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 05:02:21.989752 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0806 05:02:21.993033 (+ 3281us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 05:02:21.993795 (+ 762us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 05:02:21.993802 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 05:02:21.993807 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 05:02:21.994009 (+ 202us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 05:02:21.994463 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 05:02:21.994468 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 05:02:21.994470 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 05:02:21.995039 (+ 569us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 05:02:21.995351 (+ 312us) server_negotiation.cc:1035] Waiting for connection context
0806 05:02:21.995842 (+ 491us) server_negotiation.cc:299] Negotiation successful
0806 05:02:21.996308 (+ 466us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":596,"thread_start_us":385,"threads_started":1}
I20260806 05:02:22.000157 4748 raft_consensus.cc:504] T 00000000000000000000000000000000 P 76f2527f84ce4061a7e0161394484916 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76f2527f84ce4061a7e0161394484916" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 41365 } }
I20260806 05:02:22.000710 4748 leader_election.cc:302] T 00000000000000000000000000000000 P 76f2527f84ce4061a7e0161394484916 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 76f2527f84ce4061a7e0161394484916; no voters:
I20260806 05:02:22.001259 4748 leader_election.cc:288] T 00000000000000000000000000000000 P 76f2527f84ce4061a7e0161394484916 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 05:02:22.001394 4753 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 76f2527f84ce4061a7e0161394484916 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 05:02:22.002137 4753 raft_consensus.cc:686] T 00000000000000000000000000000000 P 76f2527f84ce4061a7e0161394484916 [term 1 LEADER]: Becoming Leader. State: Replica: 76f2527f84ce4061a7e0161394484916, State: Running, Role: LEADER
I20260806 05:02:22.002674 4753 consensus_queue.cc:237] T 00000000000000000000000000000000 P 76f2527f84ce4061a7e0161394484916 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76f2527f84ce4061a7e0161394484916" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 41365 } }
I20260806 05:02:22.003311 4748 sys_catalog.cc:564] T 00000000000000000000000000000000 P 76f2527f84ce4061a7e0161394484916 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 05:02:22.005560 4755 sys_catalog.cc:455] T 00000000000000000000000000000000 P 76f2527f84ce4061a7e0161394484916 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "76f2527f84ce4061a7e0161394484916" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76f2527f84ce4061a7e0161394484916" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 41365 } } }
I20260806 05:02:22.005630 4756 sys_catalog.cc:455] T 00000000000000000000000000000000 P 76f2527f84ce4061a7e0161394484916 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 76f2527f84ce4061a7e0161394484916. Latest consensus state: current_term: 1 leader_uuid: "76f2527f84ce4061a7e0161394484916" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76f2527f84ce4061a7e0161394484916" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 41365 } } }
I20260806 05:02:22.005985 4755 sys_catalog.cc:458] T 00000000000000000000000000000000 P 76f2527f84ce4061a7e0161394484916 [sys.catalog]: This master's current role is: LEADER
I20260806 05:02:22.006950 4756 sys_catalog.cc:458] T 00000000000000000000000000000000 P 76f2527f84ce4061a7e0161394484916 [sys.catalog]: This master's current role is: LEADER
I20260806 05:02:22.007952 4760 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 05:02:22.013484 4760 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 05:02:22.022105 4760 catalog_manager.cc:1347] Generated new cluster ID: a1a27a536f534235918cf23555d8772d
I20260806 05:02:22.022231 4760 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 05:02:22.042887 4760 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 05:02:22.044267 4760 catalog_manager.cc:1504] Loading token signing keys...
I20260806 05:02:22.053697 4760 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 76f2527f84ce4061a7e0161394484916: Generated new TSK 0
I20260806 05:02:22.054507 4760 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.9.96.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.96.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.96.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.65.keytab.
Entry for principal kudu/127.9.96.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.65.keytab.
I20260806 05:02:22.126766 9601 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
/tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785992227050689-9601-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785992227050689-9601-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785992227050689-9601-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785992227050689-9601-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.96.65:0
--local_ip_for_outbound_sockets=127.9.96.65
--webserver_interface=127.9.96.65
--webserver_port=0
--tserver_master_addrs=127.9.96.126:41365
--keytab_file=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.65.keytab
--principal=kudu/127.9.96.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.96.84:36485
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785992227050689-9601-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785992227050689-9601-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785992227050689-9601-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 05:02:22.469652 4787 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 05:02:22.469960 4787 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 05:02:22.470052 4787 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 05:02:22.477237 4787 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 05:02:22.477332 4787 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 05:02:22.477437 4787 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.96.65
I20260806 05:02:22.487016 4787 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.96.84:36485
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785992227050689-9601-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785992227050689-9601-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.65.keytab
--principal=kudu/127.9.96.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.96.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785992227050689-9601-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.96.65
--webserver_port=0
--tserver_master_addrs=127.9.96.126:41365
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.96.65
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785992227050689-9601-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 03:56:12 UTC on ccycloud.quasar-guijbm.root.comops.site
build id 4569618
ASAN enabled
I20260806 05:02:22.488200 4787 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 05:02:22.489396 4787 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 05:02:22.503167 4796 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 05:02:22.503096 4794 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 05:02:22.503386 4793 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 05:02:22.504230 4787 server_base.cc:1034] running on GCE node
I20260806 05:02:22.504873 4787 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 05:02:22.505918 4787 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 05:02:22.507153 4787 hybrid_clock.cc:648] HybridClock initialized: now 1785992542507071 us; error 53 us; skew 500 ppm
I20260806 05:02:22.507566 4787 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 05:02:22 dist-test-slave-x4qh krb5kdc[4634](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992542, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.96.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 05:02:22.510869 4787 init.cc:377] Logged in from keytab as kudu/127.9.96.65@KRBTEST.COM (short username kudu)
I20260806 05:02:22.512296 4787 webserver.cc:460] Webserver started at http://127.9.96.65:46561/ using document root <none> and password file <none>
I20260806 05:02:22.512956 4787 fs_manager.cc:359] Metadata directory not provided
I20260806 05:02:22.513092 4787 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 05:02:22.513417 4787 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 05:02:22.516402 4787 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785992227050689-9601-0/minicluster-data/ts-0/data/instance:
uuid: "bd28da1dbd8348788a31a1f389afba73"
format_stamp: "Formatted at 2026-08-06 05:02:22 on dist-test-slave-x4qh"
server_key: "9f68eab7b4d636c60cb66394d927c0b7"
server_key_iv: "8e2a7b51142987665c6356bc5a53af47"
server_key_version: "encryptionkey@0"
I20260806 05:02:22.517179 4787 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785992227050689-9601-0/minicluster-data/ts-0/wal/instance:
uuid: "bd28da1dbd8348788a31a1f389afba73"
format_stamp: "Formatted at 2026-08-06 05:02:22 on dist-test-slave-x4qh"
server_key: "9f68eab7b4d636c60cb66394d927c0b7"
server_key_iv: "8e2a7b51142987665c6356bc5a53af47"
server_key_version: "encryptionkey@0"
I20260806 05:02:22.522020 4787 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260806 05:02:22.525311 4807 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 05:02:22.526535 4787 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 05:02:22.526702 4787 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785992227050689-9601-0/minicluster-data/ts-0/data,/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785992227050689-9601-0/minicluster-data/ts-0/wal
uuid: "bd28da1dbd8348788a31a1f389afba73"
format_stamp: "Formatted at 2026-08-06 05:02:22 on dist-test-slave-x4qh"
server_key: "9f68eab7b4d636c60cb66394d927c0b7"
server_key_iv: "8e2a7b51142987665c6356bc5a53af47"
server_key_version: "encryptionkey@0"
I20260806 05:02:22.526906 4787 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785992227050689-9601-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785992227050689-9601-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785992227050689-9601-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 05:02:22.552697 4787 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 05:02:22.556636 4787 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 05:02:22.556910 4787 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 05:02:22.557869 4787 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 05:02:22.559180 4787 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 05:02:22.559248 4787 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 05:02:22.559427 4787 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 05:02:22.559502 4787 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.001s sys 0.000s
I20260806 05:02:22.593521 4787 rpc_server.cc:307] RPC server started. Bound to: 127.9.96.65:43837
I20260806 05:02:22.593685 4966 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.96.65:43837 every 8 connection(s)
I20260806 05:02:22.594887 4787 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785992227050689-9601-0/minicluster-data/ts-0/data/info.pb
I20260806 05:02:22.596916 9601 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu as pid 4787
I20260806 05:02:22.597123 9601 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785992227050689-9601-0/minicluster-data/ts-0/wal/instance
I20260806 05:02:22.597622 9601 external_mini_cluster.cc:1428] Setting key b542c09d9efc1cec269c49bef30dea9d
Aug 06 05:02:22 dist-test-slave-x4qh krb5kdc[4634](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992542, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.96.65@KRBTEST.COM for kudu/127.9.96.126@KRBTEST.COM
I20260806 05:02:22.618499 4970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:02:22.597695 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.96.65:34031 (local address 127.9.96.126:41365)
0806 05:02:22.598214 (+ 519us) server_negotiation.cc:206] Beginning negotiation
0806 05:02:22.598221 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 05:02:22.599315 (+ 1094us) server_negotiation.cc:407] Connection header received
0806 05:02:22.601913 (+ 2598us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 05:02:22.601917 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 05:02:22.601981 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 05:02:22.602072 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0806 05:02:22.604253 (+ 2181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:02:22.605111 (+ 858us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:02:22.607653 (+ 2542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:02:22.608073 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:02:22.608183 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:02:22.612485 (+ 4302us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 05:02:22.612490 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 05:02:22.612494 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 05:02:22.612527 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0806 05:02:22.615512 (+ 2985us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 05:02:22.616221 (+ 709us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 05:02:22.616226 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 05:02:22.616230 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 05:02:22.616356 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 05:02:22.616796 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 05:02:22.616801 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 05:02:22.616804 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 05:02:22.617068 (+ 264us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 05:02:22.617226 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0806 05:02:22.617901 (+ 675us) server_negotiation.cc:299] Negotiation successful
0806 05:02:22.618121 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":376,"thread_start_us":248,"threads_started":1}
I20260806 05:02:22.619503 4972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:02:22.598257 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:41365 (local address 127.9.96.65:34031)
0806 05:02:22.599140 (+ 883us) negotiation.cc:107] Waiting for socket to connect
0806 05:02:22.599189 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0806 05:02:22.601608 (+ 2419us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 05:02:22.602349 (+ 741us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 05:02:22.602363 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 05:02:22.603139 (+ 776us) client_negotiation.cc:189] Negotiated authn=SASL
0806 05:02:22.603996 (+ 857us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 05:02:22.604022 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 05:02:22.605247 (+ 1225us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 05:02:22.605254 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 05:02:22.607505 (+ 2251us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 05:02:22.607523 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 05:02:22.608320 (+ 797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 05:02:22.608329 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 05:02:22.608673 (+ 344us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:02:22.609442 (+ 769us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 05:02:22.609477 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0806 05:02:22.612250 (+ 2773us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 05:02:22.615645 (+ 3395us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 05:02:22.615654 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 05:02:22.615672 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0806 05:02:22.616087 (+ 415us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 05:02:22.616472 (+ 385us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 05:02:22.616477 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 05:02:22.616481 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 05:02:22.616670 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 05:02:22.617200 (+ 530us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 05:02:22.617216 (+ 16us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 05:02:22.617528 (+ 312us) client_negotiation.cc:769] Sending connection context
0806 05:02:22.617784 (+ 256us) client_negotiation.cc:240] Negotiation successful
0806 05:02:22.618306 (+ 522us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":606,"thread_start_us":361,"threads_started":1}
I20260806 05:02:22.621102 4967 heartbeater.cc:347] Connected to a master server at 127.9.96.126:41365
I20260806 05:02:22.621474 4967 heartbeater.cc:464] Registering TS with master...
I20260806 05:02:22.622346 4967 heartbeater.cc:511] Master 127.9.96.126:41365 requested a full tablet report, sending...
I20260806 05:02:22.624806 4689 ts_manager.cc:194] Registered new tserver with Master: bd28da1dbd8348788a31a1f389afba73 (127.9.96.65:43837)
I20260806 05:02:22.626994 4689 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.96.65@KRBTEST.COM'} at 127.9.96.65:34031
WARNING: no policy specified for kudu/127.9.96.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.96.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.96.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.66.keytab.
Entry for principal kudu/127.9.96.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.66.keytab.
I20260806 05:02:22.668292 9601 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
/tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785992227050689-9601-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785992227050689-9601-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785992227050689-9601-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785992227050689-9601-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.96.66:0
--local_ip_for_outbound_sockets=127.9.96.66
--webserver_interface=127.9.96.66
--webserver_port=0
--tserver_master_addrs=127.9.96.126:41365
--keytab_file=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.66.keytab
--principal=kudu/127.9.96.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.96.84:36485
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785992227050689-9601-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785992227050689-9601-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785992227050689-9601-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 05:02:23.008514 4976 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 05:02:23.008850 4976 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 05:02:23.009054 4976 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 05:02:23.016319 4976 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 05:02:23.016489 4976 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 05:02:23.016690 4976 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.96.66
I20260806 05:02:23.026610 4976 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.96.84:36485
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785992227050689-9601-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785992227050689-9601-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.66.keytab
--principal=kudu/127.9.96.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.96.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785992227050689-9601-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.96.66
--webserver_port=0
--tserver_master_addrs=127.9.96.126:41365
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.96.66
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785992227050689-9601-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 03:56:12 UTC on ccycloud.quasar-guijbm.root.comops.site
build id 4569618
ASAN enabled
I20260806 05:02:23.027933 4976 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 05:02:23.029294 4976 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 05:02:23.042984 4986 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 05:02:23.043043 4983 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 05:02:23.043004 4984 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 05:02:23.044447 4976 server_base.cc:1034] running on GCE node
I20260806 05:02:23.045017 4976 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 05:02:23.045919 4976 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 05:02:23.047147 4976 hybrid_clock.cc:648] HybridClock initialized: now 1785992543047111 us; error 60 us; skew 500 ppm
I20260806 05:02:23.047508 4976 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 05:02:23 dist-test-slave-x4qh krb5kdc[4634](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992543, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.96.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 05:02:23.050699 4976 init.cc:377] Logged in from keytab as kudu/127.9.96.66@KRBTEST.COM (short username kudu)
I20260806 05:02:23.052116 4976 webserver.cc:460] Webserver started at http://127.9.96.66:39945/ using document root <none> and password file <none>
I20260806 05:02:23.052778 4976 fs_manager.cc:359] Metadata directory not provided
I20260806 05:02:23.052942 4976 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 05:02:23.053238 4976 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 05:02:23.056277 4976 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785992227050689-9601-0/minicluster-data/ts-1/data/instance:
uuid: "f27001f3e7e942a996ab6cb796757e93"
format_stamp: "Formatted at 2026-08-06 05:02:23 on dist-test-slave-x4qh"
server_key: "0038f0d5882d7c907fa26999a823f7e1"
server_key_iv: "8e024a2521cc01b98eee5cab78b17a24"
server_key_version: "encryptionkey@0"
I20260806 05:02:23.057103 4976 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785992227050689-9601-0/minicluster-data/ts-1/wal/instance:
uuid: "f27001f3e7e942a996ab6cb796757e93"
format_stamp: "Formatted at 2026-08-06 05:02:23 on dist-test-slave-x4qh"
server_key: "0038f0d5882d7c907fa26999a823f7e1"
server_key_iv: "8e024a2521cc01b98eee5cab78b17a24"
server_key_version: "encryptionkey@0"
I20260806 05:02:23.062299 4976 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260806 05:02:23.065775 4997 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 05:02:23.067096 4976 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260806 05:02:23.067286 4976 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785992227050689-9601-0/minicluster-data/ts-1/data,/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785992227050689-9601-0/minicluster-data/ts-1/wal
uuid: "f27001f3e7e942a996ab6cb796757e93"
format_stamp: "Formatted at 2026-08-06 05:02:23 on dist-test-slave-x4qh"
server_key: "0038f0d5882d7c907fa26999a823f7e1"
server_key_iv: "8e024a2521cc01b98eee5cab78b17a24"
server_key_version: "encryptionkey@0"
I20260806 05:02:23.067572 4976 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785992227050689-9601-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785992227050689-9601-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785992227050689-9601-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 05:02:23.091059 4976 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 05:02:23.095500 4976 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 05:02:23.095824 4976 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 05:02:23.096853 4976 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 05:02:23.098254 4976 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 05:02:23.098335 4976 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 05:02:23.098526 4976 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 05:02:23.098590 4976 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 05:02:23.132400 4976 rpc_server.cc:307] RPC server started. Bound to: 127.9.96.66:44507
I20260806 05:02:23.132458 5164 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.96.66:44507 every 8 connection(s)
I20260806 05:02:23.133670 4976 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785992227050689-9601-0/minicluster-data/ts-1/data/info.pb
I20260806 05:02:23.135897 9601 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu as pid 4976
I20260806 05:02:23.136085 9601 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785992227050689-9601-0/minicluster-data/ts-1/wal/instance
I20260806 05:02:23.136709 9601 external_mini_cluster.cc:1428] Setting key 2a12daffa20756ba558843b38209ddcb
Aug 06 05:02:23 dist-test-slave-x4qh krb5kdc[4634](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992543, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.96.66@KRBTEST.COM for kudu/127.9.96.126@KRBTEST.COM
I20260806 05:02:23.156539 5168 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:02:23.136365 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.96.66:36445 (local address 127.9.96.126:41365)
0806 05:02:23.136889 (+ 524us) server_negotiation.cc:206] Beginning negotiation
0806 05:02:23.136894 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 05:02:23.138115 (+ 1221us) server_negotiation.cc:407] Connection header received
0806 05:02:23.140534 (+ 2419us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 05:02:23.140542 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 05:02:23.140649 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 05:02:23.140783 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0806 05:02:23.142336 (+ 1553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:02:23.143102 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:02:23.144970 (+ 1868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:02:23.145435 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:02:23.145602 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:02:23.149405 (+ 3803us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 05:02:23.149410 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 05:02:23.149414 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 05:02:23.149449 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0806 05:02:23.152971 (+ 3522us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 05:02:23.153701 (+ 730us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 05:02:23.153710 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 05:02:23.153716 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 05:02:23.153899 (+ 183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 05:02:23.154316 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 05:02:23.154325 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 05:02:23.154331 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 05:02:23.154770 (+ 439us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 05:02:23.154941 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0806 05:02:23.155704 (+ 763us) server_negotiation.cc:299] Negotiation successful
0806 05:02:23.155997 (+ 293us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":383,"thread_start_us":265,"threads_started":1}
I20260806 05:02:23.156972 5169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:02:23.136967 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:41365 (local address 127.9.96.66:36445)
0806 05:02:23.137921 (+ 954us) negotiation.cc:107] Waiting for socket to connect
0806 05:02:23.137990 (+ 69us) client_negotiation.cc:174] Beginning negotiation
0806 05:02:23.140255 (+ 2265us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 05:02:23.141031 (+ 776us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 05:02:23.141039 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 05:02:23.141560 (+ 521us) client_negotiation.cc:189] Negotiated authn=SASL
0806 05:02:23.142151 (+ 591us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 05:02:23.142169 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 05:02:23.143288 (+ 1119us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 05:02:23.143293 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 05:02:23.144774 (+ 1481us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 05:02:23.144791 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 05:02:23.145687 (+ 896us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 05:02:23.145696 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 05:02:23.145966 (+ 270us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:02:23.146718 (+ 752us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 05:02:23.146743 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0806 05:02:23.149139 (+ 2396us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 05:02:23.153119 (+ 3980us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 05:02:23.153126 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 05:02:23.153138 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0806 05:02:23.153549 (+ 411us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 05:02:23.154015 (+ 466us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 05:02:23.154018 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 05:02:23.154020 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 05:02:23.154190 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 05:02:23.154907 (+ 717us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 05:02:23.154912 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 05:02:23.155212 (+ 300us) client_negotiation.cc:769] Sending connection context
0806 05:02:23.155506 (+ 294us) client_negotiation.cc:240] Negotiation successful
0806 05:02:23.155899 (+ 393us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":670,"thread_start_us":405,"threads_started":1}
I20260806 05:02:23.158551 5165 heartbeater.cc:347] Connected to a master server at 127.9.96.126:41365
I20260806 05:02:23.158870 5165 heartbeater.cc:464] Registering TS with master...
I20260806 05:02:23.159621 5165 heartbeater.cc:511] Master 127.9.96.126:41365 requested a full tablet report, sending...
I20260806 05:02:23.161176 4689 ts_manager.cc:194] Registered new tserver with Master: f27001f3e7e942a996ab6cb796757e93 (127.9.96.66:44507)
I20260806 05:02:23.162051 4689 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.96.66@KRBTEST.COM'} at 127.9.96.66:36445
WARNING: no policy specified for kudu/127.9.96.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.96.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.96.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.67.keytab.
Entry for principal kudu/127.9.96.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.67.keytab.
I20260806 05:02:23.206131 9601 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
/tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785992227050689-9601-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785992227050689-9601-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785992227050689-9601-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785992227050689-9601-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.96.67:0
--local_ip_for_outbound_sockets=127.9.96.67
--webserver_interface=127.9.96.67
--webserver_port=0
--tserver_master_addrs=127.9.96.126:41365
--keytab_file=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.67.keytab
--principal=kudu/127.9.96.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.96.84:36485
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785992227050689-9601-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785992227050689-9601-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785992227050689-9601-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 05:02:23.542105 5172 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 05:02:23.542400 5172 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 05:02:23.542574 5172 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 05:02:23.549706 5172 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 05:02:23.549842 5172 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 05:02:23.550028 5172 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.96.67
I20260806 05:02:23.559350 5172 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.96.84:36485
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785992227050689-9601-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785992227050689-9601-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.67.keytab
--principal=kudu/127.9.96.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.96.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785992227050689-9601-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.96.67
--webserver_port=0
--tserver_master_addrs=127.9.96.126:41365
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.96.67
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785992227050689-9601-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 03:56:12 UTC on ccycloud.quasar-guijbm.root.comops.site
build id 4569618
ASAN enabled
I20260806 05:02:23.560602 5172 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 05:02:23.562031 5172 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 05:02:23.575779 5181 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 05:02:23.575719 5179 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 05:02:23.576217 5184 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 05:02:23.576861 5172 server_base.cc:1034] running on GCE node
I20260806 05:02:23.577462 5172 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 05:02:23.578388 5172 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 05:02:23.579614 5172 hybrid_clock.cc:648] HybridClock initialized: now 1785992543579562 us; error 64 us; skew 500 ppm
I20260806 05:02:23.579928 5172 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 05:02:23 dist-test-slave-x4qh krb5kdc[4634](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992543, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.96.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 05:02:23.583127 5172 init.cc:377] Logged in from keytab as kudu/127.9.96.67@KRBTEST.COM (short username kudu)
I20260806 05:02:23.584619 5172 webserver.cc:460] Webserver started at http://127.9.96.67:43555/ using document root <none> and password file <none>
I20260806 05:02:23.585238 5172 fs_manager.cc:359] Metadata directory not provided
I20260806 05:02:23.585347 5172 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 05:02:23.585605 5172 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 05:02:23.588451 5172 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785992227050689-9601-0/minicluster-data/ts-2/data/instance:
uuid: "5301c0dba3a040498611c46a060ff57f"
format_stamp: "Formatted at 2026-08-06 05:02:23 on dist-test-slave-x4qh"
server_key: "12a5303f3c8b6aec48f06ad19107c4cd"
server_key_iv: "2355949b00e65716f55c7f1433efd744"
server_key_version: "encryptionkey@0"
I20260806 05:02:23.589196 5172 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785992227050689-9601-0/minicluster-data/ts-2/wal/instance:
uuid: "5301c0dba3a040498611c46a060ff57f"
format_stamp: "Formatted at 2026-08-06 05:02:23 on dist-test-slave-x4qh"
server_key: "12a5303f3c8b6aec48f06ad19107c4cd"
server_key_iv: "2355949b00e65716f55c7f1433efd744"
server_key_version: "encryptionkey@0"
I20260806 05:02:23.594364 5172 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260806 05:02:23.597965 5195 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 05:02:23.599171 5172 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 05:02:23.599460 5172 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785992227050689-9601-0/minicluster-data/ts-2/data,/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785992227050689-9601-0/minicluster-data/ts-2/wal
uuid: "5301c0dba3a040498611c46a060ff57f"
format_stamp: "Formatted at 2026-08-06 05:02:23 on dist-test-slave-x4qh"
server_key: "12a5303f3c8b6aec48f06ad19107c4cd"
server_key_iv: "2355949b00e65716f55c7f1433efd744"
server_key_version: "encryptionkey@0"
I20260806 05:02:23.599691 5172 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785992227050689-9601-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785992227050689-9601-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785992227050689-9601-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 05:02:23.615904 5172 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 05:02:23.620033 5172 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 05:02:23.620326 5172 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 05:02:23.621204 5172 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 05:02:23.622498 5172 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 05:02:23.622610 5172 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 05:02:23.622767 5172 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 05:02:23.622856 5172 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 05:02:23.630916 4967 heartbeater.cc:503] Master 127.9.96.126:41365 was elected leader, sending a full tablet report...
I20260806 05:02:23.656888 5172 rpc_server.cc:307] RPC server started. Bound to: 127.9.96.67:46303
I20260806 05:02:23.656961 5351 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.96.67:46303 every 8 connection(s)
I20260806 05:02:23.658226 5172 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785992227050689-9601-0/minicluster-data/ts-2/data/info.pb
I20260806 05:02:23.662389 9601 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu as pid 5172
I20260806 05:02:23.662587 9601 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785992227050689-9601-0/minicluster-data/ts-2/wal/instance
I20260806 05:02:23.663040 9601 external_mini_cluster.cc:1428] Setting key 388f1a1516a140c662da40fbbb2deee7
Aug 06 05:02:23 dist-test-slave-x4qh krb5kdc[4634](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992543, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.96.67@KRBTEST.COM for kudu/127.9.96.126@KRBTEST.COM
I20260806 05:02:23.678141 5354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:02:23.660686 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.96.67:35297 (local address 127.9.96.126:41365)
0806 05:02:23.661161 (+ 475us) server_negotiation.cc:206] Beginning negotiation
0806 05:02:23.661168 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 05:02:23.662103 (+ 935us) server_negotiation.cc:407] Connection header received
0806 05:02:23.664247 (+ 2144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 05:02:23.664252 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 05:02:23.664341 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 05:02:23.664473 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0806 05:02:23.666099 (+ 1626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:02:23.666869 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:02:23.668455 (+ 1586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:02:23.668859 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:02:23.668961 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:02:23.672397 (+ 3436us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 05:02:23.672402 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 05:02:23.672405 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 05:02:23.672438 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0806 05:02:23.674956 (+ 2518us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 05:02:23.675673 (+ 717us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 05:02:23.675677 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 05:02:23.675680 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 05:02:23.675821 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 05:02:23.676200 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 05:02:23.676205 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 05:02:23.676208 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 05:02:23.676514 (+ 306us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 05:02:23.676635 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0806 05:02:23.677485 (+ 850us) server_negotiation.cc:299] Negotiation successful
0806 05:02:23.677713 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":350,"thread_start_us":220,"threads_started":1}
I20260806 05:02:23.679016 5356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:02:23.661227 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:41365 (local address 127.9.96.67:35297)
0806 05:02:23.661925 (+ 698us) negotiation.cc:107] Waiting for socket to connect
0806 05:02:23.661973 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0806 05:02:23.663926 (+ 1953us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 05:02:23.664660 (+ 734us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 05:02:23.664667 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 05:02:23.665264 (+ 597us) client_negotiation.cc:189] Negotiated authn=SASL
0806 05:02:23.665904 (+ 640us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 05:02:23.665933 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 05:02:23.666997 (+ 1064us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 05:02:23.667001 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 05:02:23.668312 (+ 1311us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 05:02:23.668326 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 05:02:23.669046 (+ 720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 05:02:23.669051 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 05:02:23.669235 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:02:23.669920 (+ 685us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 05:02:23.669946 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0806 05:02:23.672220 (+ 2274us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 05:02:23.675080 (+ 2860us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 05:02:23.675087 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 05:02:23.675097 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0806 05:02:23.675566 (+ 469us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 05:02:23.675914 (+ 348us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 05:02:23.675917 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 05:02:23.675919 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 05:02:23.676082 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 05:02:23.676643 (+ 561us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 05:02:23.676651 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 05:02:23.677052 (+ 401us) client_negotiation.cc:769] Sending connection context
0806 05:02:23.677368 (+ 316us) client_negotiation.cc:240] Negotiation successful
0806 05:02:23.677864 (+ 496us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":480,"thread_start_us":282,"threads_started":1}
I20260806 05:02:23.680383 5352 heartbeater.cc:347] Connected to a master server at 127.9.96.126:41365
I20260806 05:02:23.680709 5352 heartbeater.cc:464] Registering TS with master...
I20260806 05:02:23.681429 5352 heartbeater.cc:511] Master 127.9.96.126:41365 requested a full tablet report, sending...
I20260806 05:02:23.682790 4689 ts_manager.cc:194] Registered new tserver with Master: 5301c0dba3a040498611c46a060ff57f (127.9.96.67:46303)
I20260806 05:02:23.683652 4689 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.96.67@KRBTEST.COM'} at 127.9.96.67:35297
I20260806 05:02:23.690371 9601 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 05:02:23.708431 5354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:02:23.694460 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42542 (local address 127.9.96.126:41365)
0806 05:02:23.694608 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0806 05:02:23.694613 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 05:02:23.695123 (+ 510us) server_negotiation.cc:407] Connection header received
0806 05:02:23.695314 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 05:02:23.695323 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 05:02:23.695462 (+ 139us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 05:02:23.695566 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0806 05:02:23.696988 (+ 1422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:02:23.697733 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:02:23.699273 (+ 1540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:02:23.699755 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:02:23.699907 (+ 152us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:02:23.702599 (+ 2692us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 05:02:23.702605 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 05:02:23.702610 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 05:02:23.702650 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0806 05:02:23.705431 (+ 2781us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 05:02:23.706248 (+ 817us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 05:02:23.706254 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 05:02:23.706261 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 05:02:23.706434 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 05:02:23.706909 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 05:02:23.706914 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 05:02:23.706917 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 05:02:23.707213 (+ 296us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 05:02:23.707530 (+ 317us) server_negotiation.cc:1035] Waiting for connection context
0806 05:02:23.707921 (+ 391us) server_negotiation.cc:299] Negotiation successful
0806 05:02:23.708121 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260806 05:02:23.712831 4689 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42542:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 05:02:23.719770 4689 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 05:02:23.757637 5380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:02:23.733219 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54288 (local address 127.9.96.65:43837)
0806 05:02:23.743541 (+ 10322us) server_negotiation.cc:206] Beginning negotiation
0806 05:02:23.743554 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 05:02:23.749245 (+ 5691us) server_negotiation.cc:407] Connection header received
0806 05:02:23.749580 (+ 335us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 05:02:23.749591 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 05:02:23.749862 (+ 271us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 05:02:23.750068 (+ 206us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 05:02:23.750852 (+ 784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:02:23.752655 (+ 1803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:02:23.754972 (+ 2317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:02:23.756118 (+ 1146us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:02:23.756242 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:02:23.756482 (+ 240us) server_negotiation.cc:1035] Waiting for connection context
0806 05:02:23.757100 (+ 618us) server_negotiation.cc:299] Negotiation successful
0806 05:02:23.757289 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10036,"thread_start_us":9809,"threads_started":1}
I20260806 05:02:23.757712 5376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:02:23.732664 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.65:43837 (local address 127.0.0.1:54288)
0806 05:02:23.749110 (+ 16446us) negotiation.cc:107] Waiting for socket to connect
0806 05:02:23.749130 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0806 05:02:23.749362 (+ 232us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 05:02:23.750166 (+ 804us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 05:02:23.750178 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 05:02:23.750219 (+ 41us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 05:02:23.750658 (+ 439us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 05:02:23.750675 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 05:02:23.752806 (+ 2131us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 05:02:23.752811 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 05:02:23.754810 (+ 1999us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 05:02:23.754823 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 05:02:23.756615 (+ 1792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 05:02:23.756621 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 05:02:23.756854 (+ 233us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:02:23.756905 (+ 51us) client_negotiation.cc:769] Sending connection context
0806 05:02:23.757046 (+ 141us) client_negotiation.cc:240] Negotiation successful
0806 05:02:23.757247 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":16305,"mutex_wait_us":48,"thread_start_us":17038,"threads_started":1}
I20260806 05:02:23.758920 5383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:02:23.734969 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33500 (local address 127.9.96.67:46303)
0806 05:02:23.749551 (+ 14582us) server_negotiation.cc:206] Beginning negotiation
0806 05:02:23.749560 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 05:02:23.750297 (+ 737us) server_negotiation.cc:407] Connection header received
0806 05:02:23.750412 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 05:02:23.750420 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 05:02:23.750637 (+ 217us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 05:02:23.750829 (+ 192us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 05:02:23.751652 (+ 823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:02:23.753224 (+ 1572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:02:23.756064 (+ 2840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:02:23.757569 (+ 1505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:02:23.757714 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:02:23.758236 (+ 522us) server_negotiation.cc:1035] Waiting for connection context
0806 05:02:23.758408 (+ 172us) server_negotiation.cc:299] Negotiation successful
0806 05:02:23.758561 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14373,"thread_start_us":14217,"threads_started":1}
I20260806 05:02:23.759487 5378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:02:23.734261 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.67:46303 (local address 127.0.0.1:33500)
0806 05:02:23.750038 (+ 15777us) negotiation.cc:107] Waiting for socket to connect
0806 05:02:23.750056 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 05:02:23.750172 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 05:02:23.750896 (+ 724us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 05:02:23.750901 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 05:02:23.750930 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 05:02:23.751351 (+ 421us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 05:02:23.751364 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 05:02:23.753342 (+ 1978us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 05:02:23.753349 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 05:02:23.755741 (+ 2392us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 05:02:23.755756 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 05:02:23.758000 (+ 2244us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 05:02:23.758006 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 05:02:23.758194 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:02:23.758240 (+ 46us) client_negotiation.cc:769] Sending connection context
0806 05:02:23.758357 (+ 117us) client_negotiation.cc:240] Negotiation successful
0806 05:02:23.758476 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15619,"mutex_wait_us":1,"thread_start_us":12943,"threads_started":1}
I20260806 05:02:23.759552 5382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:02:23.732535 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.66:44507 (local address 127.0.0.1:33714)
0806 05:02:23.747697 (+ 15162us) negotiation.cc:107] Waiting for socket to connect
0806 05:02:23.747760 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0806 05:02:23.747979 (+ 219us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 05:02:23.751307 (+ 3328us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 05:02:23.751312 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 05:02:23.751335 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 05:02:23.751763 (+ 428us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 05:02:23.751777 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 05:02:23.753704 (+ 1927us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 05:02:23.753709 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 05:02:23.755971 (+ 2262us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 05:02:23.756004 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 05:02:23.758347 (+ 2343us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 05:02:23.758353 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 05:02:23.758507 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:02:23.758537 (+ 30us) client_negotiation.cc:769] Sending connection context
0806 05:02:23.758612 (+ 75us) client_negotiation.cc:240] Negotiation successful
0806 05:02:23.758777 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14839,"mutex_wait_us":24,"thread_start_us":16269,"threads_started":1}
I20260806 05:02:23.759954 5377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:02:23.732678 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33714 (local address 127.9.96.66:44507)
0806 05:02:23.750734 (+ 18056us) server_negotiation.cc:206] Beginning negotiation
0806 05:02:23.750745 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 05:02:23.750774 (+ 29us) server_negotiation.cc:407] Connection header received
0806 05:02:23.750866 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 05:02:23.750875 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 05:02:23.751099 (+ 224us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 05:02:23.751310 (+ 211us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 05:02:23.751957 (+ 647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:02:23.753546 (+ 1589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:02:23.756186 (+ 2640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:02:23.757667 (+ 1481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:02:23.759172 (+ 1505us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:02:23.759331 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0806 05:02:23.759458 (+ 127us) server_negotiation.cc:299] Negotiation successful
0806 05:02:23.759631 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17888,"thread_start_us":17798,"threads_started":1}
I20260806 05:02:23.762023 4873 tablet_service.cc:1467] Processing CreateTablet for tablet 0bb84b0075734846bf6eee513423b99e (DEFAULT_TABLE table=test-table [id=ec5c41a5835649a5bb8f8e769a0966b3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 05:02:23.763264 4873 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0bb84b0075734846bf6eee513423b99e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 05:02:23.762993 5259 tablet_service.cc:1467] Processing CreateTablet for tablet 0bb84b0075734846bf6eee513423b99e (DEFAULT_TABLE table=test-table [id=ec5c41a5835649a5bb8f8e769a0966b3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 05:02:23.764148 5259 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0bb84b0075734846bf6eee513423b99e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 05:02:23.769836 5067 tablet_service.cc:1467] Processing CreateTablet for tablet 0bb84b0075734846bf6eee513423b99e (DEFAULT_TABLE table=test-table [id=ec5c41a5835649a5bb8f8e769a0966b3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 05:02:23.771062 5067 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0bb84b0075734846bf6eee513423b99e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 05:02:23.776242 5385 tablet_bootstrap.cc:492] T 0bb84b0075734846bf6eee513423b99e P bd28da1dbd8348788a31a1f389afba73: Bootstrap starting.
I20260806 05:02:23.779176 5385 tablet_bootstrap.cc:654] T 0bb84b0075734846bf6eee513423b99e P bd28da1dbd8348788a31a1f389afba73: Neither blocks nor log segments found. Creating new log.
I20260806 05:02:23.779625 5386 tablet_bootstrap.cc:492] T 0bb84b0075734846bf6eee513423b99e P 5301c0dba3a040498611c46a060ff57f: Bootstrap starting.
I20260806 05:02:23.780553 5385 log.cc:824] T 0bb84b0075734846bf6eee513423b99e P bd28da1dbd8348788a31a1f389afba73: Log is configured to *not* fsync() on all Append() calls
I20260806 05:02:23.783056 5386 tablet_bootstrap.cc:654] T 0bb84b0075734846bf6eee513423b99e P 5301c0dba3a040498611c46a060ff57f: Neither blocks nor log segments found. Creating new log.
I20260806 05:02:23.784300 5386 log.cc:824] T 0bb84b0075734846bf6eee513423b99e P 5301c0dba3a040498611c46a060ff57f: Log is configured to *not* fsync() on all Append() calls
I20260806 05:02:23.784364 5385 tablet_bootstrap.cc:492] T 0bb84b0075734846bf6eee513423b99e P bd28da1dbd8348788a31a1f389afba73: No bootstrap required, opened a new log
I20260806 05:02:23.784739 5385 ts_tablet_manager.cc:1397] T 0bb84b0075734846bf6eee513423b99e P bd28da1dbd8348788a31a1f389afba73: Time spent bootstrapping tablet: real 0.009s user 0.002s sys 0.005s
I20260806 05:02:23.785389 5388 tablet_bootstrap.cc:492] T 0bb84b0075734846bf6eee513423b99e P f27001f3e7e942a996ab6cb796757e93: Bootstrap starting.
I20260806 05:02:23.788247 5388 tablet_bootstrap.cc:654] T 0bb84b0075734846bf6eee513423b99e P f27001f3e7e942a996ab6cb796757e93: Neither blocks nor log segments found. Creating new log.
I20260806 05:02:23.789224 5388 log.cc:824] T 0bb84b0075734846bf6eee513423b99e P f27001f3e7e942a996ab6cb796757e93: Log is configured to *not* fsync() on all Append() calls
I20260806 05:02:23.789937 5386 tablet_bootstrap.cc:492] T 0bb84b0075734846bf6eee513423b99e P 5301c0dba3a040498611c46a060ff57f: No bootstrap required, opened a new log
I20260806 05:02:23.790189 5386 ts_tablet_manager.cc:1397] T 0bb84b0075734846bf6eee513423b99e P 5301c0dba3a040498611c46a060ff57f: Time spent bootstrapping tablet: real 0.011s user 0.005s sys 0.003s
I20260806 05:02:23.791875 5388 tablet_bootstrap.cc:492] T 0bb84b0075734846bf6eee513423b99e P f27001f3e7e942a996ab6cb796757e93: No bootstrap required, opened a new log
I20260806 05:02:23.792174 5388 ts_tablet_manager.cc:1397] T 0bb84b0075734846bf6eee513423b99e P f27001f3e7e942a996ab6cb796757e93: Time spent bootstrapping tablet: real 0.007s user 0.006s sys 0.000s
I20260806 05:02:23.796423 5385 raft_consensus.cc:348] T 0bb84b0075734846bf6eee513423b99e P bd28da1dbd8348788a31a1f389afba73 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f27001f3e7e942a996ab6cb796757e93" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 44507 } } peers { permanent_uuid: "bd28da1dbd8348788a31a1f389afba73" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 43837 } } peers { permanent_uuid: "5301c0dba3a040498611c46a060ff57f" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 46303 } }
I20260806 05:02:23.796834 5385 raft_consensus.cc:374] T 0bb84b0075734846bf6eee513423b99e P bd28da1dbd8348788a31a1f389afba73 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 05:02:23.796976 5385 raft_consensus.cc:729] T 0bb84b0075734846bf6eee513423b99e P bd28da1dbd8348788a31a1f389afba73 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd28da1dbd8348788a31a1f389afba73, State: Initialized, Role: FOLLOWER
I20260806 05:02:23.797636 5385 consensus_queue.cc:260] T 0bb84b0075734846bf6eee513423b99e P bd28da1dbd8348788a31a1f389afba73 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f27001f3e7e942a996ab6cb796757e93" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 44507 } } peers { permanent_uuid: "bd28da1dbd8348788a31a1f389afba73" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 43837 } } peers { permanent_uuid: "5301c0dba3a040498611c46a060ff57f" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 46303 } }
I20260806 05:02:23.799363 5385 ts_tablet_manager.cc:1428] T 0bb84b0075734846bf6eee513423b99e P bd28da1dbd8348788a31a1f389afba73: Time spent starting tablet: real 0.014s user 0.009s sys 0.003s
I20260806 05:02:23.800591 5388 raft_consensus.cc:348] T 0bb84b0075734846bf6eee513423b99e P f27001f3e7e942a996ab6cb796757e93 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f27001f3e7e942a996ab6cb796757e93" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 44507 } } peers { permanent_uuid: "bd28da1dbd8348788a31a1f389afba73" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 43837 } } peers { permanent_uuid: "5301c0dba3a040498611c46a060ff57f" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 46303 } }
I20260806 05:02:23.800622 5386 raft_consensus.cc:348] T 0bb84b0075734846bf6eee513423b99e P 5301c0dba3a040498611c46a060ff57f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f27001f3e7e942a996ab6cb796757e93" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 44507 } } peers { permanent_uuid: "bd28da1dbd8348788a31a1f389afba73" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 43837 } } peers { permanent_uuid: "5301c0dba3a040498611c46a060ff57f" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 46303 } }
I20260806 05:02:23.800925 5388 raft_consensus.cc:374] T 0bb84b0075734846bf6eee513423b99e P f27001f3e7e942a996ab6cb796757e93 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 05:02:23.801031 5386 raft_consensus.cc:374] T 0bb84b0075734846bf6eee513423b99e P 5301c0dba3a040498611c46a060ff57f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 05:02:23.801052 5388 raft_consensus.cc:729] T 0bb84b0075734846bf6eee513423b99e P f27001f3e7e942a996ab6cb796757e93 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f27001f3e7e942a996ab6cb796757e93, State: Initialized, Role: FOLLOWER
I20260806 05:02:23.801144 5386 raft_consensus.cc:729] T 0bb84b0075734846bf6eee513423b99e P 5301c0dba3a040498611c46a060ff57f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5301c0dba3a040498611c46a060ff57f, State: Initialized, Role: FOLLOWER
I20260806 05:02:23.801610 5388 consensus_queue.cc:260] T 0bb84b0075734846bf6eee513423b99e P f27001f3e7e942a996ab6cb796757e93 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f27001f3e7e942a996ab6cb796757e93" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 44507 } } peers { permanent_uuid: "bd28da1dbd8348788a31a1f389afba73" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 43837 } } peers { permanent_uuid: "5301c0dba3a040498611c46a060ff57f" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 46303 } }
I20260806 05:02:23.801837 5386 consensus_queue.cc:260] T 0bb84b0075734846bf6eee513423b99e P 5301c0dba3a040498611c46a060ff57f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f27001f3e7e942a996ab6cb796757e93" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 44507 } } peers { permanent_uuid: "bd28da1dbd8348788a31a1f389afba73" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 43837 } } peers { permanent_uuid: "5301c0dba3a040498611c46a060ff57f" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 46303 } }
I20260806 05:02:23.802649 5165 heartbeater.cc:503] Master 127.9.96.126:41365 was elected leader, sending a full tablet report...
I20260806 05:02:23.803509 5388 ts_tablet_manager.cc:1428] T 0bb84b0075734846bf6eee513423b99e P f27001f3e7e942a996ab6cb796757e93: Time spent starting tablet: real 0.011s user 0.008s sys 0.004s
I20260806 05:02:23.804584 5352 heartbeater.cc:503] Master 127.9.96.126:41365 was elected leader, sending a full tablet report...
I20260806 05:02:23.805409 5386 ts_tablet_manager.cc:1428] T 0bb84b0075734846bf6eee513423b99e P 5301c0dba3a040498611c46a060ff57f: Time spent starting tablet: real 0.015s user 0.010s sys 0.005s
I20260806 05:02:23.820139 5396 raft_consensus.cc:482] T 0bb84b0075734846bf6eee513423b99e P 5301c0dba3a040498611c46a060ff57f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 05:02:23.820451 5396 raft_consensus.cc:504] T 0bb84b0075734846bf6eee513423b99e P 5301c0dba3a040498611c46a060ff57f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f27001f3e7e942a996ab6cb796757e93" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 44507 } } peers { permanent_uuid: "bd28da1dbd8348788a31a1f389afba73" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 43837 } } peers { permanent_uuid: "5301c0dba3a040498611c46a060ff57f" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 46303 } }
I20260806 05:02:23.822047 5396 leader_election.cc:288] T 0bb84b0075734846bf6eee513423b99e P 5301c0dba3a040498611c46a060ff57f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f27001f3e7e942a996ab6cb796757e93 (127.9.96.66:44507), bd28da1dbd8348788a31a1f389afba73 (127.9.96.65:43837)
I20260806 05:02:23.828831 5356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:02:23.822573 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.65:43837 (local address 127.9.96.67:46267)
0806 05:02:23.822954 (+ 381us) negotiation.cc:107] Waiting for socket to connect
0806 05:02:23.822981 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0806 05:02:23.823143 (+ 162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 05:02:23.823769 (+ 626us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 05:02:23.823772 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 05:02:23.823793 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 05:02:23.824008 (+ 215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 05:02:23.824021 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 05:02:23.825098 (+ 1077us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 05:02:23.825104 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 05:02:23.826584 (+ 1480us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 05:02:23.826597 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 05:02:23.827974 (+ 1377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 05:02:23.827980 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 05:02:23.828185 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:02:23.828220 (+ 35us) client_negotiation.cc:769] Sending connection context
0806 05:02:23.828328 (+ 108us) client_negotiation.cc:240] Negotiation successful
0806 05:02:23.828477 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":234,"mutex_wait_us":1}
I20260806 05:02:23.828877 5380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:02:23.822516 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.96.67:46267 (local address 127.9.96.65:43837)
0806 05:02:23.822735 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0806 05:02:23.822742 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 05:02:23.823295 (+ 553us) server_negotiation.cc:407] Connection header received
0806 05:02:23.823403 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 05:02:23.823408 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 05:02:23.823490 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 05:02:23.823576 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 05:02:23.824185 (+ 609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:02:23.824967 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:02:23.826749 (+ 1782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:02:23.827819 (+ 1070us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:02:23.827923 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:02:23.827997 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0806 05:02:23.828385 (+ 388us) server_negotiation.cc:299] Negotiation successful
0806 05:02:23.828555 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260806 05:02:23.829547 5377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:02:23.822597 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.96.67:45867 (local address 127.9.96.66:44507)
0806 05:02:23.822795 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0806 05:02:23.822800 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 05:02:23.823568 (+ 768us) server_negotiation.cc:407] Connection header received
0806 05:02:23.823748 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 05:02:23.823753 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 05:02:23.823828 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 05:02:23.823926 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 05:02:23.824587 (+ 661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:02:23.825462 (+ 875us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:02:23.827355 (+ 1893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:02:23.828368 (+ 1013us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:02:23.828489 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:02:23.828574 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0806 05:02:23.829184 (+ 610us) server_negotiation.cc:299] Negotiation successful
0806 05:02:23.829307 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260806 05:02:23.829579 5397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:02:23.822621 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.66:44507 (local address 127.9.96.67:45867)
0806 05:02:23.823456 (+ 835us) negotiation.cc:107] Waiting for socket to connect
0806 05:02:23.823479 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0806 05:02:23.823603 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 05:02:23.823976 (+ 373us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 05:02:23.823981 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 05:02:23.824006 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 05:02:23.824413 (+ 407us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 05:02:23.824424 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 05:02:23.825615 (+ 1191us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 05:02:23.825622 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 05:02:23.827199 (+ 1577us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 05:02:23.827213 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 05:02:23.828593 (+ 1380us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 05:02:23.828599 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 05:02:23.828833 (+ 234us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:02:23.828872 (+ 39us) client_negotiation.cc:769] Sending connection context
0806 05:02:23.829129 (+ 257us) client_negotiation.cc:240] Negotiation successful
0806 05:02:23.829255 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":698,"thread_start_us":444,"threads_started":1}
I20260806 05:02:23.829921 4899 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0bb84b0075734846bf6eee513423b99e" candidate_uuid: "5301c0dba3a040498611c46a060ff57f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd28da1dbd8348788a31a1f389afba73" is_pre_election: true
I20260806 05:02:23.830605 4899 raft_consensus.cc:2454] T 0bb84b0075734846bf6eee513423b99e P bd28da1dbd8348788a31a1f389afba73 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5301c0dba3a040498611c46a060ff57f in term 0.
I20260806 05:02:23.830675 5094 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0bb84b0075734846bf6eee513423b99e" candidate_uuid: "5301c0dba3a040498611c46a060ff57f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f27001f3e7e942a996ab6cb796757e93" is_pre_election: true
I20260806 05:02:23.831231 5094 raft_consensus.cc:2454] T 0bb84b0075734846bf6eee513423b99e P f27001f3e7e942a996ab6cb796757e93 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5301c0dba3a040498611c46a060ff57f in term 0.
I20260806 05:02:23.831419 5200 leader_election.cc:302] T 0bb84b0075734846bf6eee513423b99e P 5301c0dba3a040498611c46a060ff57f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5301c0dba3a040498611c46a060ff57f, bd28da1dbd8348788a31a1f389afba73; no voters:
I20260806 05:02:23.831853 5396 raft_consensus.cc:2781] T 0bb84b0075734846bf6eee513423b99e P 5301c0dba3a040498611c46a060ff57f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 05:02:23.831995 5396 raft_consensus.cc:482] T 0bb84b0075734846bf6eee513423b99e P 5301c0dba3a040498611c46a060ff57f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 05:02:23.832127 5396 raft_consensus.cc:3037] T 0bb84b0075734846bf6eee513423b99e P 5301c0dba3a040498611c46a060ff57f [term 0 FOLLOWER]: Advancing to term 1
I20260806 05:02:23.834317 5396 raft_consensus.cc:504] T 0bb84b0075734846bf6eee513423b99e P 5301c0dba3a040498611c46a060ff57f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f27001f3e7e942a996ab6cb796757e93" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 44507 } } peers { permanent_uuid: "bd28da1dbd8348788a31a1f389afba73" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 43837 } } peers { permanent_uuid: "5301c0dba3a040498611c46a060ff57f" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 46303 } }
I20260806 05:02:23.834909 5396 leader_election.cc:288] T 0bb84b0075734846bf6eee513423b99e P 5301c0dba3a040498611c46a060ff57f [CANDIDATE]: Term 1 election: Requested vote from peers f27001f3e7e942a996ab6cb796757e93 (127.9.96.66:44507), bd28da1dbd8348788a31a1f389afba73 (127.9.96.65:43837)
I20260806 05:02:23.835412 5094 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0bb84b0075734846bf6eee513423b99e" candidate_uuid: "5301c0dba3a040498611c46a060ff57f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f27001f3e7e942a996ab6cb796757e93"
I20260806 05:02:23.835496 4899 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0bb84b0075734846bf6eee513423b99e" candidate_uuid: "5301c0dba3a040498611c46a060ff57f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd28da1dbd8348788a31a1f389afba73"
I20260806 05:02:23.835611 5094 raft_consensus.cc:3037] T 0bb84b0075734846bf6eee513423b99e P f27001f3e7e942a996ab6cb796757e93 [term 0 FOLLOWER]: Advancing to term 1
I20260806 05:02:23.835767 4899 raft_consensus.cc:3037] T 0bb84b0075734846bf6eee513423b99e P bd28da1dbd8348788a31a1f389afba73 [term 0 FOLLOWER]: Advancing to term 1
I20260806 05:02:23.837904 5094 raft_consensus.cc:2454] T 0bb84b0075734846bf6eee513423b99e P f27001f3e7e942a996ab6cb796757e93 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5301c0dba3a040498611c46a060ff57f in term 1.
I20260806 05:02:23.838315 5202 leader_election.cc:302] T 0bb84b0075734846bf6eee513423b99e P 5301c0dba3a040498611c46a060ff57f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5301c0dba3a040498611c46a060ff57f, f27001f3e7e942a996ab6cb796757e93; no voters:
I20260806 05:02:23.838487 4899 raft_consensus.cc:2454] T 0bb84b0075734846bf6eee513423b99e P bd28da1dbd8348788a31a1f389afba73 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5301c0dba3a040498611c46a060ff57f in term 1.
I20260806 05:02:23.838676 5396 raft_consensus.cc:2781] T 0bb84b0075734846bf6eee513423b99e P 5301c0dba3a040498611c46a060ff57f [term 1 FOLLOWER]: Leader election won for term 1
I20260806 05:02:23.839178 5396 raft_consensus.cc:686] T 0bb84b0075734846bf6eee513423b99e P 5301c0dba3a040498611c46a060ff57f [term 1 LEADER]: Becoming Leader. State: Replica: 5301c0dba3a040498611c46a060ff57f, State: Running, Role: LEADER
I20260806 05:02:23.839725 5396 consensus_queue.cc:237] T 0bb84b0075734846bf6eee513423b99e P 5301c0dba3a040498611c46a060ff57f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f27001f3e7e942a996ab6cb796757e93" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 44507 } } peers { permanent_uuid: "bd28da1dbd8348788a31a1f389afba73" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 43837 } } peers { permanent_uuid: "5301c0dba3a040498611c46a060ff57f" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 46303 } }
I20260806 05:02:23.844609 4689 catalog_manager.cc:5495] T 0bb84b0075734846bf6eee513423b99e P 5301c0dba3a040498611c46a060ff57f reported cstate change: term changed from 0 to 1, leader changed from <none> to 5301c0dba3a040498611c46a060ff57f (127.9.96.67). New cstate: current_term: 1 leader_uuid: "5301c0dba3a040498611c46a060ff57f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f27001f3e7e942a996ab6cb796757e93" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 44507 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bd28da1dbd8348788a31a1f389afba73" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 43837 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5301c0dba3a040498611c46a060ff57f" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 46303 } health_report { overall_health: HEALTHY } } }
W20260806 05:02:23.847517 4969 tablet.cc:2369] T 0bb84b0075734846bf6eee513423b99e P bd28da1dbd8348788a31a1f389afba73: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 05:02:23.880837 5383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:02:23.875030 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33506 (local address 127.9.96.67:46303)
0806 05:02:23.875310 (+ 280us) server_negotiation.cc:206] Beginning negotiation
0806 05:02:23.875318 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 05:02:23.875464 (+ 146us) server_negotiation.cc:407] Connection header received
0806 05:02:23.875711 (+ 247us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 05:02:23.875715 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 05:02:23.875796 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 05:02:23.875967 (+ 171us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 05:02:23.876384 (+ 417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:02:23.877121 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:02:23.878511 (+ 1390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:02:23.878925 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:02:23.879084 (+ 159us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:02:23.879755 (+ 671us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 05:02:23.880233 (+ 478us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 05:02:23.880430 (+ 197us) server_negotiation.cc:1035] Waiting for connection context
0806 05:02:23.880560 (+ 130us) server_negotiation.cc:299] Negotiation successful
0806 05:02:23.880608 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":109}
W20260806 05:02:23.885663 5166 tablet.cc:2369] T 0bb84b0075734846bf6eee513423b99e P f27001f3e7e942a996ab6cb796757e93: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 05:02:23.888608 4899 raft_consensus.cc:1261] T 0bb84b0075734846bf6eee513423b99e P bd28da1dbd8348788a31a1f389afba73 [term 1 FOLLOWER]: Refusing update from remote peer 5301c0dba3a040498611c46a060ff57f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 05:02:23.888710 5094 raft_consensus.cc:1261] T 0bb84b0075734846bf6eee513423b99e P f27001f3e7e942a996ab6cb796757e93 [term 1 FOLLOWER]: Refusing update from remote peer 5301c0dba3a040498611c46a060ff57f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 05:02:23.889684 5399 consensus_queue.cc:1035] T 0bb84b0075734846bf6eee513423b99e P 5301c0dba3a040498611c46a060ff57f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f27001f3e7e942a996ab6cb796757e93" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 44507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 05:02:23.890074 5396 consensus_queue.cc:1035] T 0bb84b0075734846bf6eee513423b99e P 5301c0dba3a040498611c46a060ff57f [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd28da1dbd8348788a31a1f389afba73" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 43837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 05:02:23.901803 5403 mvcc.cc:205] Tried to move back new op lower bound from 7315425459758321664 to 7315425459571613696. Current Snapshot: MvccSnapshot[applied={T|T < 7315425459758321664}]
I20260806 05:02:23.903252 5404 mvcc.cc:205] Tried to move back new op lower bound from 7315425459758321664 to 7315425459571613696. Current Snapshot: MvccSnapshot[applied={T|T < 7315425459758321664}]
I20260806 05:02:23.918676 4688 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42542:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 05:02:23.919001 4688 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42542:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 05:02:23.922612 4688 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 76f2527f84ce4061a7e0161394484916: Sending DeleteTablet for 3 replicas of tablet 0bb84b0075734846bf6eee513423b99e
I20260806 05:02:23.923815 4873 tablet_service.cc:1514] Processing DeleteTablet for tablet 0bb84b0075734846bf6eee513423b99e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 05:02:23 UTC) from {username='kudu', principal='kudu/127.9.96.126@KRBTEST.COM'} at 127.0.0.1:54288
I20260806 05:02:23.923801 5067 tablet_service.cc:1514] Processing DeleteTablet for tablet 0bb84b0075734846bf6eee513423b99e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 05:02:23 UTC) from {username='kudu', principal='kudu/127.9.96.126@KRBTEST.COM'} at 127.0.0.1:33714
I20260806 05:02:23.924741 5414 tablet_replica.cc:330] T 0bb84b0075734846bf6eee513423b99e P bd28da1dbd8348788a31a1f389afba73: stopping tablet replica
I20260806 05:02:23.925205 5414 raft_consensus.cc:2229] T 0bb84b0075734846bf6eee513423b99e P bd28da1dbd8348788a31a1f389afba73 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 05:02:23.925247 5259 tablet_service.cc:1514] Processing DeleteTablet for tablet 0bb84b0075734846bf6eee513423b99e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 05:02:23 UTC) from {username='kudu', principal='kudu/127.9.96.126@KRBTEST.COM'} at 127.0.0.1:33500
I20260806 05:02:23.925830 5414 raft_consensus.cc:2258] T 0bb84b0075734846bf6eee513423b99e P bd28da1dbd8348788a31a1f389afba73 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 05:02:23.927134 9601 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu pid 4787
I20260806 05:02:23.928503 5414 ts_tablet_manager.cc:1905] T 0bb84b0075734846bf6eee513423b99e P bd28da1dbd8348788a31a1f389afba73: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 05:02:23.934921 5416 tablet_replica.cc:330] T 0bb84b0075734846bf6eee513423b99e P 5301c0dba3a040498611c46a060ff57f: stopping tablet replica
I20260806 05:02:23.935088 5415 tablet_replica.cc:330] T 0bb84b0075734846bf6eee513423b99e P f27001f3e7e942a996ab6cb796757e93: stopping tablet replica
I20260806 05:02:23.935518 5415 raft_consensus.cc:2229] T 0bb84b0075734846bf6eee513423b99e P f27001f3e7e942a996ab6cb796757e93 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 05:02:23.935545 5416 raft_consensus.cc:2229] T 0bb84b0075734846bf6eee513423b99e P 5301c0dba3a040498611c46a060ff57f [term 1 LEADER]: Raft consensus shutting down.
I20260806 05:02:23.936018 5415 raft_consensus.cc:2258] T 0bb84b0075734846bf6eee513423b99e P f27001f3e7e942a996ab6cb796757e93 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 05:02:23.936358 5416 raft_consensus.cc:2258] T 0bb84b0075734846bf6eee513423b99e P 5301c0dba3a040498611c46a060ff57f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 05:02:23.937584 5414 ts_tablet_manager.cc:1918] T 0bb84b0075734846bf6eee513423b99e P bd28da1dbd8348788a31a1f389afba73: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 05:02:23.937839 5414 log.cc:1196] T 0bb84b0075734846bf6eee513423b99e P bd28da1dbd8348788a31a1f389afba73: Deleting WAL directory at /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785992227050689-9601-0/minicluster-data/ts-0/wal/wals/0bb84b0075734846bf6eee513423b99e
I20260806 05:02:23.938482 5414 ts_tablet_manager.cc:1939] T 0bb84b0075734846bf6eee513423b99e P bd28da1dbd8348788a31a1f389afba73: Deleting consensus metadata
I20260806 05:02:23.938601 5416 ts_tablet_manager.cc:1905] T 0bb84b0075734846bf6eee513423b99e P 5301c0dba3a040498611c46a060ff57f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 05:02:23.939113 5415 ts_tablet_manager.cc:1905] T 0bb84b0075734846bf6eee513423b99e P f27001f3e7e942a996ab6cb796757e93: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 05:02:23.940531 4675 catalog_manager.cc:4841] TS bd28da1dbd8348788a31a1f389afba73 (127.9.96.65:43837): tablet 0bb84b0075734846bf6eee513423b99e (table test-table [id=ec5c41a5835649a5bb8f8e769a0966b3]) successfully deleted
I20260806 05:02:23.946513 5416 ts_tablet_manager.cc:1918] T 0bb84b0075734846bf6eee513423b99e P 5301c0dba3a040498611c46a060ff57f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 05:02:23.946661 5416 log.cc:1196] T 0bb84b0075734846bf6eee513423b99e P 5301c0dba3a040498611c46a060ff57f: Deleting WAL directory at /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785992227050689-9601-0/minicluster-data/ts-2/wal/wals/0bb84b0075734846bf6eee513423b99e
I20260806 05:02:23.947227 5416 ts_tablet_manager.cc:1939] T 0bb84b0075734846bf6eee513423b99e P 5301c0dba3a040498611c46a060ff57f: Deleting consensus metadata
I20260806 05:02:23.948985 4674 catalog_manager.cc:4841] TS 5301c0dba3a040498611c46a060ff57f (127.9.96.67:46303): tablet 0bb84b0075734846bf6eee513423b99e (table test-table [id=ec5c41a5835649a5bb8f8e769a0966b3]) successfully deleted
I20260806 05:02:23.949412 5415 ts_tablet_manager.cc:1918] T 0bb84b0075734846bf6eee513423b99e P f27001f3e7e942a996ab6cb796757e93: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 05:02:23.949594 5415 log.cc:1196] T 0bb84b0075734846bf6eee513423b99e P f27001f3e7e942a996ab6cb796757e93: Deleting WAL directory at /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785992227050689-9601-0/minicluster-data/ts-1/wal/wals/0bb84b0075734846bf6eee513423b99e
I20260806 05:02:23.950155 5415 ts_tablet_manager.cc:1939] T 0bb84b0075734846bf6eee513423b99e P f27001f3e7e942a996ab6cb796757e93: Deleting consensus metadata
I20260806 05:02:23.951606 4676 catalog_manager.cc:4841] TS f27001f3e7e942a996ab6cb796757e93 (127.9.96.66:44507): tablet 0bb84b0075734846bf6eee513423b99e (table test-table [id=ec5c41a5835649a5bb8f8e769a0966b3]) successfully deleted
Aug 06 05:02:23 dist-test-slave-x4qh krb5kdc[4634](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992541, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.96.65@KRBTEST.COM
I20260806 05:02:23.966845 5380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:02:23.930368 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54296 (local address 127.9.96.65:43837)
0806 05:02:23.930615 (+ 247us) server_negotiation.cc:206] Beginning negotiation
0806 05:02:23.930624 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 05:02:23.943912 (+ 13288us) server_negotiation.cc:407] Connection header received
0806 05:02:23.944223 (+ 311us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 05:02:23.944229 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 05:02:23.944324 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 05:02:23.944415 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0806 05:02:23.946313 (+ 1898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:02:23.947459 (+ 1146us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:02:23.949733 (+ 2274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:02:23.950389 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:02:23.950490 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:02:23.959690 (+ 9200us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 05:02:23.959707 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 05:02:23.959725 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 05:02:23.959775 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0806 05:02:23.963596 (+ 3821us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 05:02:23.964530 (+ 934us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 05:02:23.964541 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 05:02:23.964545 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 05:02:23.964708 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 05:02:23.965115 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 05:02:23.965121 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 05:02:23.965125 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 05:02:23.965580 (+ 455us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 05:02:23.965900 (+ 320us) server_negotiation.cc:1035] Waiting for connection context
0806 05:02:23.966230 (+ 330us) server_negotiation.cc:299] Negotiation successful
0806 05:02:23.966456 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260806 05:02:23.967563 4956 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 05:02:24.088639 9601 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu with pid 4787
I20260806 05:02:24.116535 9601 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu pid 4976
Aug 06 05:02:24 dist-test-slave-x4qh krb5kdc[4634](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992541, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.96.66@KRBTEST.COM
I20260806 05:02:24.134518 5377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:02:24.117345 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33726 (local address 127.9.96.66:44507)
0806 05:02:24.117550 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0806 05:02:24.117556 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 05:02:24.117580 (+ 24us) server_negotiation.cc:407] Connection header received
0806 05:02:24.117736 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 05:02:24.117745 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 05:02:24.117835 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 05:02:24.117955 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0806 05:02:24.119249 (+ 1294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:02:24.120539 (+ 1290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:02:24.121854 (+ 1315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:02:24.122555 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:02:24.122695 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:02:24.127299 (+ 4604us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 05:02:24.127318 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 05:02:24.127339 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 05:02:24.127415 (+ 76us) server_negotiation.cc:922] calling sasl_server_start()
0806 05:02:24.131443 (+ 4028us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 05:02:24.132245 (+ 802us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 05:02:24.132254 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 05:02:24.132258 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 05:02:24.132405 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 05:02:24.132794 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 05:02:24.132799 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 05:02:24.132802 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 05:02:24.133229 (+ 427us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 05:02:24.133540 (+ 311us) server_negotiation.cc:1035] Waiting for connection context
0806 05:02:24.133870 (+ 330us) server_negotiation.cc:299] Negotiation successful
0806 05:02:24.134113 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260806 05:02:24.135279 5157 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 05:02:24.226326 9601 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu with pid 4976
I20260806 05:02:24.248059 9601 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu pid 5172
Aug 06 05:02:24 dist-test-slave-x4qh krb5kdc[4634](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992541, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.96.67@KRBTEST.COM
I20260806 05:02:24.265640 5383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:02:24.248905 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33514 (local address 127.9.96.67:46303)
0806 05:02:24.249377 (+ 472us) server_negotiation.cc:206] Beginning negotiation
0806 05:02:24.249401 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0806 05:02:24.249415 (+ 14us) server_negotiation.cc:407] Connection header received
0806 05:02:24.249467 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 05:02:24.249471 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 05:02:24.249543 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 05:02:24.249676 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0806 05:02:24.251016 (+ 1340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:02:24.251921 (+ 905us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:02:24.253281 (+ 1360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:02:24.253746 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:02:24.253891 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:02:24.259255 (+ 5364us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 05:02:24.259267 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 05:02:24.259282 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 05:02:24.259329 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0806 05:02:24.262640 (+ 3311us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 05:02:24.263510 (+ 870us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 05:02:24.263517 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 05:02:24.263520 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 05:02:24.263693 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 05:02:24.264134 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 05:02:24.264137 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 05:02:24.264139 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 05:02:24.264493 (+ 354us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 05:02:24.264760 (+ 267us) server_negotiation.cc:1035] Waiting for connection context
0806 05:02:24.265109 (+ 349us) server_negotiation.cc:299] Negotiation successful
0806 05:02:24.265271 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":343}
I20260806 05:02:24.266530 5345 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 05:02:24.392477 9601 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu with pid 5172
I20260806 05:02:24.415648 9601 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu pid 4651
I20260806 05:02:24.416805 4736 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 05:02:24.530969 9601 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu with pid 4651
2026-08-06T05:02:24Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (4685 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-08-06T05:02:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T05:02:24Z Disabled control of system clock
I20260806 05:02:24.623162 9601 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
/tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785992227050689-9601-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785992227050689-9601-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785992227050689-9601-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785992227050689-9601-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.96.126:41191
--webserver_interface=127.9.96.126
--webserver_port=0
--builtin_ntp_servers=127.9.96.84:35917
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.96.126:41191
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260806 05:02:24.959021 5425 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 05:02:24.959311 5425 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 05:02:24.959470 5425 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 05:02:24.967496 5425 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 05:02:24.967612 5425 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 05:02:24.967676 5425 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 05:02:24.967759 5425 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 05:02:24.967834 5425 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 05:02:24.977643 5425 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.96.84:35917
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785992227050689-9601-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785992227050689-9601-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.96.126:41191
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.96.126:41191
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785992227050689-9601-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.96.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785992227050689-9601-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 03:56:12 UTC on ccycloud.quasar-guijbm.root.comops.site
build id 4569618
ASAN enabled
I20260806 05:02:24.979027 5425 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 05:02:24.980940 5425 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 05:02:24.992636 5430 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 05:02:24.992743 5431 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 05:02:24.992770 5433 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 05:02:24.993921 5425 server_base.cc:1034] running on GCE node
I20260806 05:02:24.994779 5425 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 05:02:24.996268 5425 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 05:02:24.997529 5425 hybrid_clock.cc:648] HybridClock initialized: now 1785992544997490 us; error 53 us; skew 500 ppm
I20260806 05:02:24.997920 5425 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 05:02:25.000416 5425 webserver.cc:460] Webserver started at http://127.9.96.126:37075/ using document root <none> and password file <none>
I20260806 05:02:25.001166 5425 fs_manager.cc:359] Metadata directory not provided
I20260806 05:02:25.001307 5425 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 05:02:25.001672 5425 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 05:02:25.004750 5425 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785992227050689-9601-0/minicluster-data/master-0/data/instance:
uuid: "24d4fe668b6748b7a3016815025a8830"
format_stamp: "Formatted at 2026-08-06 05:02:24 on dist-test-slave-x4qh"
server_key: "d2de87e614223978bb5848b9b60e16ed"
server_key_iv: "729b6de1ccac2abc0770561f757b808a"
server_key_version: "encryptionkey@0"
I20260806 05:02:25.005591 5425 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785992227050689-9601-0/minicluster-data/master-0/wal/instance:
uuid: "24d4fe668b6748b7a3016815025a8830"
format_stamp: "Formatted at 2026-08-06 05:02:24 on dist-test-slave-x4qh"
server_key: "d2de87e614223978bb5848b9b60e16ed"
server_key_iv: "729b6de1ccac2abc0770561f757b808a"
server_key_version: "encryptionkey@0"
I20260806 05:02:25.010962 5425 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260806 05:02:25.014595 5440 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 05:02:25.016002 5425 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 05:02:25.016207 5425 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785992227050689-9601-0/minicluster-data/master-0/data,/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785992227050689-9601-0/minicluster-data/master-0/wal
uuid: "24d4fe668b6748b7a3016815025a8830"
format_stamp: "Formatted at 2026-08-06 05:02:24 on dist-test-slave-x4qh"
server_key: "d2de87e614223978bb5848b9b60e16ed"
server_key_iv: "729b6de1ccac2abc0770561f757b808a"
server_key_version: "encryptionkey@0"
I20260806 05:02:25.016443 5425 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785992227050689-9601-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785992227050689-9601-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785992227050689-9601-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 05:02:25.026772 5425 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 05:02:25.027609 5425 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 05:02:25.027900 5425 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 05:02:25.046788 5425 rpc_server.cc:307] RPC server started. Bound to: 127.9.96.126:41191
I20260806 05:02:25.046829 5526 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.96.126:41191 every 8 connection(s)
I20260806 05:02:25.048115 5425 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785992227050689-9601-0/minicluster-data/master-0/data/info.pb
I20260806 05:02:25.051687 5527 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 05:02:25.058609 9601 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu as pid 5425
I20260806 05:02:25.058817 9601 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785992227050689-9601-0/minicluster-data/master-0/wal/instance
I20260806 05:02:25.059319 9601 external_mini_cluster.cc:1428] Setting key f8f4adcc3e081352917262939c243cc7
I20260806 05:02:25.061903 5527 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 24d4fe668b6748b7a3016815025a8830: Bootstrap starting.
I20260806 05:02:25.065155 5527 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 24d4fe668b6748b7a3016815025a8830: Neither blocks nor log segments found. Creating new log.
I20260806 05:02:25.066285 5527 log.cc:824] T 00000000000000000000000000000000 P 24d4fe668b6748b7a3016815025a8830: Log is configured to *not* fsync() on all Append() calls
I20260806 05:02:25.069469 5527 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 24d4fe668b6748b7a3016815025a8830: No bootstrap required, opened a new log
I20260806 05:02:25.073807 5530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:02:25.061191 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57156 (local address 127.9.96.126:41191)
0806 05:02:25.062251 (+ 1060us) server_negotiation.cc:206] Beginning negotiation
0806 05:02:25.062262 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 05:02:25.062305 (+ 43us) server_negotiation.cc:407] Connection header received
0806 05:02:25.064115 (+ 1810us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 05:02:25.064137 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 05:02:25.064572 (+ 435us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 05:02:25.064872 (+ 300us) server_negotiation.cc:226] Negotiated authn=SASL
0806 05:02:25.065529 (+ 657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:02:25.066916 (+ 1387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:02:25.068478 (+ 1562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:02:25.069596 (+ 1118us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:02:25.069786 (+ 190us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:02:25.070886 (+ 1100us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 05:02:25.070897 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 05:02:25.070913 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 05:02:25.071680 (+ 767us) server_negotiation.cc:922] calling sasl_server_start()
0806 05:02:25.071798 (+ 118us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 05:02:25.071828 (+ 30us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 05:02:25.071947 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0806 05:02:25.072178 (+ 231us) server_negotiation.cc:299] Negotiation successful
0806 05:02:25.072509 (+ 331us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":608,"thread_start_us":340,"threads_started":1}
I20260806 05:02:25.079125 5527 raft_consensus.cc:348] T 00000000000000000000000000000000 P 24d4fe668b6748b7a3016815025a8830 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "24d4fe668b6748b7a3016815025a8830" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 41191 } }
I20260806 05:02:25.079486 5527 raft_consensus.cc:374] T 00000000000000000000000000000000 P 24d4fe668b6748b7a3016815025a8830 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 05:02:25.079567 5527 raft_consensus.cc:729] T 00000000000000000000000000000000 P 24d4fe668b6748b7a3016815025a8830 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 24d4fe668b6748b7a3016815025a8830, State: Initialized, Role: FOLLOWER
I20260806 05:02:25.080030 5527 consensus_queue.cc:260] T 00000000000000000000000000000000 P 24d4fe668b6748b7a3016815025a8830 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "24d4fe668b6748b7a3016815025a8830" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 41191 } }
I20260806 05:02:25.080235 5527 raft_consensus.cc:388] T 00000000000000000000000000000000 P 24d4fe668b6748b7a3016815025a8830 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 05:02:25.080332 5527 raft_consensus.cc:482] T 00000000000000000000000000000000 P 24d4fe668b6748b7a3016815025a8830 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 05:02:25.080461 5527 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 24d4fe668b6748b7a3016815025a8830 [term 0 FOLLOWER]: Advancing to term 1
I20260806 05:02:25.082782 5527 raft_consensus.cc:504] T 00000000000000000000000000000000 P 24d4fe668b6748b7a3016815025a8830 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "24d4fe668b6748b7a3016815025a8830" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 41191 } }
I20260806 05:02:25.083202 5527 leader_election.cc:302] T 00000000000000000000000000000000 P 24d4fe668b6748b7a3016815025a8830 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 24d4fe668b6748b7a3016815025a8830; no voters:
I20260806 05:02:25.083798 5527 leader_election.cc:288] T 00000000000000000000000000000000 P 24d4fe668b6748b7a3016815025a8830 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 05:02:25.083958 5532 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 24d4fe668b6748b7a3016815025a8830 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 05:02:25.084702 5532 raft_consensus.cc:686] T 00000000000000000000000000000000 P 24d4fe668b6748b7a3016815025a8830 [term 1 LEADER]: Becoming Leader. State: Replica: 24d4fe668b6748b7a3016815025a8830, State: Running, Role: LEADER
I20260806 05:02:25.085199 5532 consensus_queue.cc:237] T 00000000000000000000000000000000 P 24d4fe668b6748b7a3016815025a8830 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "24d4fe668b6748b7a3016815025a8830" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 41191 } }
I20260806 05:02:25.085858 5527 sys_catalog.cc:564] T 00000000000000000000000000000000 P 24d4fe668b6748b7a3016815025a8830 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 05:02:25.088153 5534 sys_catalog.cc:455] T 00000000000000000000000000000000 P 24d4fe668b6748b7a3016815025a8830 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "24d4fe668b6748b7a3016815025a8830" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "24d4fe668b6748b7a3016815025a8830" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 41191 } } }
I20260806 05:02:25.088954 5534 sys_catalog.cc:458] T 00000000000000000000000000000000 P 24d4fe668b6748b7a3016815025a8830 [sys.catalog]: This master's current role is: LEADER
I20260806 05:02:25.088943 5535 sys_catalog.cc:455] T 00000000000000000000000000000000 P 24d4fe668b6748b7a3016815025a8830 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 24d4fe668b6748b7a3016815025a8830. Latest consensus state: current_term: 1 leader_uuid: "24d4fe668b6748b7a3016815025a8830" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "24d4fe668b6748b7a3016815025a8830" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 41191 } } }
I20260806 05:02:25.091958 5535 sys_catalog.cc:458] T 00000000000000000000000000000000 P 24d4fe668b6748b7a3016815025a8830 [sys.catalog]: This master's current role is: LEADER
I20260806 05:02:25.092344 5547 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 05:02:25.098986 5547 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 05:02:25.108094 5547 catalog_manager.cc:1347] Generated new cluster ID: b7aa1e2c04cf4fd4a0cc7a678cc09c00
I20260806 05:02:25.108208 5547 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 05:02:25.134414 5547 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 05:02:25.136225 5547 catalog_manager.cc:1504] Loading token signing keys...
I20260806 05:02:25.145740 5547 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 24d4fe668b6748b7a3016815025a8830: Generated new TSK 0
I20260806 05:02:25.146695 5547 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 05:02:25.150712 9601 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
/tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785992227050689-9601-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785992227050689-9601-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785992227050689-9601-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785992227050689-9601-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.96.65:0
--local_ip_for_outbound_sockets=127.9.96.65
--webserver_interface=127.9.96.65
--webserver_port=0
--tserver_master_addrs=127.9.96.126:41191
--builtin_ntp_servers=127.9.96.84:35917
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260806 05:02:25.491498 5559 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 05:02:25.491828 5559 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 05:02:25.491998 5559 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 05:02:25.499135 5559 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 05:02:25.499281 5559 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 05:02:25.499517 5559 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.96.65
I20260806 05:02:25.511408 5559 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.96.84:35917
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785992227050689-9601-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785992227050689-9601-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.96.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785992227050689-9601-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.96.65
--webserver_port=0
--tserver_master_addrs=127.9.96.126:41191
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.96.65
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785992227050689-9601-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 03:56:12 UTC on ccycloud.quasar-guijbm.root.comops.site
build id 4569618
ASAN enabled
I20260806 05:02:25.512985 5559 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 05:02:25.515458 5559 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 05:02:25.529039 5569 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 05:02:25.529003 5566 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 05:02:25.529292 5565 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 05:02:25.529937 5559 server_base.cc:1034] running on GCE node
I20260806 05:02:25.530591 5559 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 05:02:25.531628 5559 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 05:02:25.532872 5559 hybrid_clock.cc:648] HybridClock initialized: now 1785992545532816 us; error 43 us; skew 500 ppm
I20260806 05:02:25.533205 5559 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 05:02:25.535472 5559 webserver.cc:460] Webserver started at http://127.9.96.65:46369/ using document root <none> and password file <none>
I20260806 05:02:25.536161 5559 fs_manager.cc:359] Metadata directory not provided
I20260806 05:02:25.536298 5559 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 05:02:25.536633 5559 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 05:02:25.539489 5559 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785992227050689-9601-0/minicluster-data/ts-0/data/instance:
uuid: "0278739c639545aaa978f098bde50717"
format_stamp: "Formatted at 2026-08-06 05:02:25 on dist-test-slave-x4qh"
server_key: "de49c679e6da964befae2c7ddce514b8"
server_key_iv: "1b7fd6055c5d71f50fb081ddfc2f7c34"
server_key_version: "encryptionkey@0"
I20260806 05:02:25.540292 5559 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785992227050689-9601-0/minicluster-data/ts-0/wal/instance:
uuid: "0278739c639545aaa978f098bde50717"
format_stamp: "Formatted at 2026-08-06 05:02:25 on dist-test-slave-x4qh"
server_key: "de49c679e6da964befae2c7ddce514b8"
server_key_iv: "1b7fd6055c5d71f50fb081ddfc2f7c34"
server_key_version: "encryptionkey@0"
I20260806 05:02:25.545163 5559 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260806 05:02:25.548636 5575 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 05:02:25.549973 5559 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.004s
I20260806 05:02:25.550145 5559 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785992227050689-9601-0/minicluster-data/ts-0/data,/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785992227050689-9601-0/minicluster-data/ts-0/wal
uuid: "0278739c639545aaa978f098bde50717"
format_stamp: "Formatted at 2026-08-06 05:02:25 on dist-test-slave-x4qh"
server_key: "de49c679e6da964befae2c7ddce514b8"
server_key_iv: "1b7fd6055c5d71f50fb081ddfc2f7c34"
server_key_version: "encryptionkey@0"
I20260806 05:02:25.550391 5559 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785992227050689-9601-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785992227050689-9601-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785992227050689-9601-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 05:02:25.574292 5559 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 05:02:25.575299 5559 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 05:02:25.575618 5559 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 05:02:25.576629 5559 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 05:02:25.578222 5559 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 05:02:25.578354 5559 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 05:02:25.578482 5559 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 05:02:25.578576 5559 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 05:02:25.613171 5559 rpc_server.cc:307] RPC server started. Bound to: 127.9.96.65:41859
I20260806 05:02:25.613269 5745 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.96.65:41859 every 8 connection(s)
I20260806 05:02:25.614488 5559 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785992227050689-9601-0/minicluster-data/ts-0/data/info.pb
I20260806 05:02:25.620411 9601 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu as pid 5559
I20260806 05:02:25.620620 9601 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785992227050689-9601-0/minicluster-data/ts-0/wal/instance
I20260806 05:02:25.621280 9601 external_mini_cluster.cc:1428] Setting key f463ec53ccf0bc61c5840657f6cf3e92
I20260806 05:02:25.624116 9601 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
/tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785992227050689-9601-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785992227050689-9601-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785992227050689-9601-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785992227050689-9601-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.96.66:0
--local_ip_for_outbound_sockets=127.9.96.66
--webserver_interface=127.9.96.66
--webserver_port=0
--tserver_master_addrs=127.9.96.126:41191
--builtin_ntp_servers=127.9.96.84:35917
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260806 05:02:25.628197 5748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:02:25.617106 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.96.65:46447 (local address 127.9.96.126:41191)
0806 05:02:25.617559 (+ 453us) server_negotiation.cc:206] Beginning negotiation
0806 05:02:25.617566 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 05:02:25.618808 (+ 1242us) server_negotiation.cc:407] Connection header received
0806 05:02:25.620754 (+ 1946us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 05:02:25.620760 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 05:02:25.620872 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 05:02:25.620996 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0806 05:02:25.622245 (+ 1249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:02:25.623135 (+ 890us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:02:25.624890 (+ 1755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:02:25.625670 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:02:25.625857 (+ 187us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:02:25.627153 (+ 1296us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 05:02:25.627170 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 05:02:25.627177 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 05:02:25.627203 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0806 05:02:25.627260 (+ 57us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 05:02:25.627278 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 05:02:25.627402 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0806 05:02:25.627645 (+ 243us) server_negotiation.cc:299] Negotiation successful
0806 05:02:25.627761 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":321,"thread_start_us":209,"threads_started":1}
I20260806 05:02:25.628888 5749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:02:25.617717 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:41191 (local address 127.9.96.65:46447)
0806 05:02:25.618622 (+ 905us) negotiation.cc:107] Waiting for socket to connect
0806 05:02:25.618666 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0806 05:02:25.620427 (+ 1761us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 05:02:25.621234 (+ 807us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 05:02:25.621242 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 05:02:25.621354 (+ 112us) client_negotiation.cc:189] Negotiated authn=SASL
0806 05:02:25.622074 (+ 720us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 05:02:25.622109 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 05:02:25.623316 (+ 1207us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 05:02:25.623320 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 05:02:25.624732 (+ 1412us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 05:02:25.624746 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 05:02:25.625916 (+ 1170us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 05:02:25.625921 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 05:02:25.626109 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:02:25.626843 (+ 734us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 05:02:25.626845 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0806 05:02:25.626884 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 05:02:25.626961 (+ 77us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 05:02:25.627434 (+ 473us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 05:02:25.627443 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 05:02:25.627449 (+ 6us) client_negotiation.cc:769] Sending connection context
0806 05:02:25.627590 (+ 141us) client_negotiation.cc:240] Negotiation successful
0806 05:02:25.627883 (+ 293us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":493,"thread_start_us":276,"threads_started":1}
I20260806 05:02:25.630337 5746 heartbeater.cc:347] Connected to a master server at 127.9.96.126:41191
I20260806 05:02:25.630635 5746 heartbeater.cc:464] Registering TS with master...
I20260806 05:02:25.631345 5746 heartbeater.cc:511] Master 127.9.96.126:41191 requested a full tablet report, sending...
I20260806 05:02:25.633450 5472 ts_manager.cc:194] Registered new tserver with Master: 0278739c639545aaa978f098bde50717 (127.9.96.65:41859)
I20260806 05:02:25.634938 5472 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.9.96.65:46447
W20260806 05:02:25.965839 5750 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 05:02:25.966104 5750 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 05:02:25.966192 5750 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 05:02:25.973376 5750 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 05:02:25.973475 5750 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 05:02:25.973647 5750 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.96.66
I20260806 05:02:25.982718 5750 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.96.84:35917
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785992227050689-9601-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785992227050689-9601-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.96.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785992227050689-9601-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.96.66
--webserver_port=0
--tserver_master_addrs=127.9.96.126:41191
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.96.66
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785992227050689-9601-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 03:56:12 UTC on ccycloud.quasar-guijbm.root.comops.site
build id 4569618
ASAN enabled
I20260806 05:02:25.984054 5750 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 05:02:25.985824 5750 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 05:02:25.999776 5757 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 05:02:25.999851 5760 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 05:02:25.999991 5764 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 05:02:26.005127 5750 server_base.cc:1034] running on GCE node
I20260806 05:02:26.005867 5750 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 05:02:26.006843 5750 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 05:02:26.008108 5750 hybrid_clock.cc:648] HybridClock initialized: now 1785992546008069 us; error 58 us; skew 500 ppm
I20260806 05:02:26.008455 5750 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 05:02:26.010804 5750 webserver.cc:460] Webserver started at http://127.9.96.66:37249/ using document root <none> and password file <none>
I20260806 05:02:26.011564 5750 fs_manager.cc:359] Metadata directory not provided
I20260806 05:02:26.011698 5750 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 05:02:26.012033 5750 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 05:02:26.014983 5750 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785992227050689-9601-0/minicluster-data/ts-1/data/instance:
uuid: "4468f796ea43458986fb979632e3f125"
format_stamp: "Formatted at 2026-08-06 05:02:26 on dist-test-slave-x4qh"
server_key: "9fc1af8dc612e43fbffc8275b57f8968"
server_key_iv: "5e463c10dfc32a4b3b089c27190379d2"
server_key_version: "encryptionkey@0"
I20260806 05:02:26.015815 5750 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785992227050689-9601-0/minicluster-data/ts-1/wal/instance:
uuid: "4468f796ea43458986fb979632e3f125"
format_stamp: "Formatted at 2026-08-06 05:02:26 on dist-test-slave-x4qh"
server_key: "9fc1af8dc612e43fbffc8275b57f8968"
server_key_iv: "5e463c10dfc32a4b3b089c27190379d2"
server_key_version: "encryptionkey@0"
I20260806 05:02:26.021041 5750 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260806 05:02:26.024519 5773 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 05:02:26.025799 5750 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260806 05:02:26.025985 5750 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785992227050689-9601-0/minicluster-data/ts-1/data,/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785992227050689-9601-0/minicluster-data/ts-1/wal
uuid: "4468f796ea43458986fb979632e3f125"
format_stamp: "Formatted at 2026-08-06 05:02:26 on dist-test-slave-x4qh"
server_key: "9fc1af8dc612e43fbffc8275b57f8968"
server_key_iv: "5e463c10dfc32a4b3b089c27190379d2"
server_key_version: "encryptionkey@0"
I20260806 05:02:26.026218 5750 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785992227050689-9601-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785992227050689-9601-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785992227050689-9601-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 05:02:26.044204 5750 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 05:02:26.044975 5750 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 05:02:26.045274 5750 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 05:02:26.046259 5750 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 05:02:26.047760 5750 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 05:02:26.047891 5750 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 05:02:26.048022 5750 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 05:02:26.048121 5750 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 05:02:26.085122 5750 rpc_server.cc:307] RPC server started. Bound to: 127.9.96.66:39247
I20260806 05:02:26.085248 5934 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.96.66:39247 every 8 connection(s)
I20260806 05:02:26.086572 5750 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785992227050689-9601-0/minicluster-data/ts-1/data/info.pb
I20260806 05:02:26.095789 9601 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu as pid 5750
I20260806 05:02:26.095975 9601 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785992227050689-9601-0/minicluster-data/ts-1/wal/instance
I20260806 05:02:26.096446 9601 external_mini_cluster.cc:1428] Setting key b5eb85a7ec38ce1595d6a85f9f55a342
I20260806 05:02:26.099318 9601 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
/tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785992227050689-9601-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785992227050689-9601-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785992227050689-9601-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785992227050689-9601-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.96.67:0
--local_ip_for_outbound_sockets=127.9.96.67
--webserver_interface=127.9.96.67
--webserver_port=0
--tserver_master_addrs=127.9.96.126:41191
--builtin_ntp_servers=127.9.96.84:35917
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260806 05:02:26.099817 5748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:02:26.089486 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.96.66:42211 (local address 127.9.96.126:41191)
0806 05:02:26.089785 (+ 299us) server_negotiation.cc:206] Beginning negotiation
0806 05:02:26.089789 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 05:02:26.091144 (+ 1355us) server_negotiation.cc:407] Connection header received
0806 05:02:26.093004 (+ 1860us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 05:02:26.093009 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 05:02:26.093101 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 05:02:26.093216 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0806 05:02:26.094262 (+ 1046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:02:26.094997 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:02:26.096850 (+ 1853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:02:26.097418 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:02:26.097559 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:02:26.098991 (+ 1432us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 05:02:26.098996 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 05:02:26.098999 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 05:02:26.099017 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0806 05:02:26.099047 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 05:02:26.099059 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 05:02:26.099157 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0806 05:02:26.099419 (+ 262us) server_negotiation.cc:299] Negotiation successful
0806 05:02:26.099517 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":110}
I20260806 05:02:26.100780 5941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:02:26.090068 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:41191 (local address 127.9.96.66:42211)
0806 05:02:26.090984 (+ 916us) negotiation.cc:107] Waiting for socket to connect
0806 05:02:26.091019 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0806 05:02:26.092726 (+ 1707us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 05:02:26.093458 (+ 732us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 05:02:26.093467 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 05:02:26.093546 (+ 79us) client_negotiation.cc:189] Negotiated authn=SASL
0806 05:02:26.094091 (+ 545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 05:02:26.094106 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 05:02:26.095161 (+ 1055us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 05:02:26.095172 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 05:02:26.096690 (+ 1518us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 05:02:26.096703 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 05:02:26.097645 (+ 942us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 05:02:26.097652 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 05:02:26.097887 (+ 235us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:02:26.098641 (+ 754us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 05:02:26.098644 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0806 05:02:26.098683 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 05:02:26.098762 (+ 79us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 05:02:26.099157 (+ 395us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 05:02:26.099171 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 05:02:26.099181 (+ 10us) client_negotiation.cc:769] Sending connection context
0806 05:02:26.099377 (+ 196us) client_negotiation.cc:240] Negotiation successful
0806 05:02:26.099792 (+ 415us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":497,"thread_start_us":291,"threads_started":1}
I20260806 05:02:26.102185 5936 heartbeater.cc:347] Connected to a master server at 127.9.96.126:41191
I20260806 05:02:26.102515 5936 heartbeater.cc:464] Registering TS with master...
I20260806 05:02:26.103446 5936 heartbeater.cc:511] Master 127.9.96.126:41191 requested a full tablet report, sending...
I20260806 05:02:26.104983 5472 ts_manager.cc:194] Registered new tserver with Master: 4468f796ea43458986fb979632e3f125 (127.9.96.66:39247)
I20260806 05:02:26.105849 5472 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.9.96.66:42211
W20260806 05:02:26.435259 5942 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 05:02:26.435546 5942 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 05:02:26.435712 5942 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 05:02:26.442719 5942 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 05:02:26.442807 5942 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 05:02:26.442974 5942 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.96.67
I20260806 05:02:26.453049 5942 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.96.84:35917
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785992227050689-9601-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785992227050689-9601-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.96.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785992227050689-9601-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.96.67
--webserver_port=0
--tserver_master_addrs=127.9.96.126:41191
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.96.67
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785992227050689-9601-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 03:56:12 UTC on ccycloud.quasar-guijbm.root.comops.site
build id 4569618
ASAN enabled
I20260806 05:02:26.454389 5942 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 05:02:26.456265 5942 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 05:02:26.469800 5951 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 05:02:26.469781 5952 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 05:02:26.470419 5958 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 05:02:26.472757 5942 server_base.cc:1034] running on GCE node
I20260806 05:02:26.473325 5942 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 05:02:26.474524 5942 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 05:02:26.475764 5942 hybrid_clock.cc:648] HybridClock initialized: now 1785992546475716 us; error 42 us; skew 500 ppm
I20260806 05:02:26.476073 5942 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 05:02:26.478286 5942 webserver.cc:460] Webserver started at http://127.9.96.67:36119/ using document root <none> and password file <none>
I20260806 05:02:26.478976 5942 fs_manager.cc:359] Metadata directory not provided
I20260806 05:02:26.479063 5942 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 05:02:26.479354 5942 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 05:02:26.482271 5942 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785992227050689-9601-0/minicluster-data/ts-2/data/instance:
uuid: "e18e0d8cca2946beb025190c17a0312f"
format_stamp: "Formatted at 2026-08-06 05:02:26 on dist-test-slave-x4qh"
server_key: "a0120f7bf2ccbabdbec2e7c86da91645"
server_key_iv: "5603d6a1ecc30fd06b018613b523990b"
server_key_version: "encryptionkey@0"
I20260806 05:02:26.483009 5942 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785992227050689-9601-0/minicluster-data/ts-2/wal/instance:
uuid: "e18e0d8cca2946beb025190c17a0312f"
format_stamp: "Formatted at 2026-08-06 05:02:26 on dist-test-slave-x4qh"
server_key: "a0120f7bf2ccbabdbec2e7c86da91645"
server_key_iv: "5603d6a1ecc30fd06b018613b523990b"
server_key_version: "encryptionkey@0"
I20260806 05:02:26.488139 5942 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.002s
I20260806 05:02:26.491819 5966 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 05:02:26.493144 5942 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260806 05:02:26.493312 5942 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785992227050689-9601-0/minicluster-data/ts-2/data,/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785992227050689-9601-0/minicluster-data/ts-2/wal
uuid: "e18e0d8cca2946beb025190c17a0312f"
format_stamp: "Formatted at 2026-08-06 05:02:26 on dist-test-slave-x4qh"
server_key: "a0120f7bf2ccbabdbec2e7c86da91645"
server_key_iv: "5603d6a1ecc30fd06b018613b523990b"
server_key_version: "encryptionkey@0"
I20260806 05:02:26.493533 5942 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785992227050689-9601-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785992227050689-9601-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785992227050689-9601-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 05:02:26.511338 5942 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 05:02:26.512231 5942 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 05:02:26.512531 5942 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 05:02:26.513548 5942 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 05:02:26.515059 5942 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 05:02:26.515190 5942 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 05:02:26.515329 5942 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 05:02:26.515448 5942 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.001s sys 0.000s
I20260806 05:02:26.551927 5942 rpc_server.cc:307] RPC server started. Bound to: 127.9.96.67:45113
I20260806 05:02:26.552023 6126 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.96.67:45113 every 8 connection(s)
I20260806 05:02:26.553253 5942 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785992227050689-9601-0/minicluster-data/ts-2/data/info.pb
I20260806 05:02:26.560927 9601 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu as pid 5942
I20260806 05:02:26.561115 9601 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785992227050689-9601-0/minicluster-data/ts-2/wal/instance
I20260806 05:02:26.561594 9601 external_mini_cluster.cc:1428] Setting key 8a382551d8e6909794e8cde247833c6f
I20260806 05:02:26.567006 5748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:02:26.556283 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.96.67:40779 (local address 127.9.96.126:41191)
0806 05:02:26.556480 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0806 05:02:26.556486 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 05:02:26.557981 (+ 1495us) server_negotiation.cc:407] Connection header received
0806 05:02:26.559816 (+ 1835us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 05:02:26.559821 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 05:02:26.559920 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 05:02:26.560044 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0806 05:02:26.561165 (+ 1121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:02:26.562026 (+ 861us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:02:26.563987 (+ 1961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:02:26.564555 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:02:26.564725 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:02:26.566143 (+ 1418us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 05:02:26.566149 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 05:02:26.566154 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 05:02:26.566181 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0806 05:02:26.566220 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 05:02:26.566232 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 05:02:26.566344 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0806 05:02:26.566564 (+ 220us) server_negotiation.cc:299] Negotiation successful
0806 05:02:26.566666 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260806 05:02:26.568095 6132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:02:26.556850 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:41191 (local address 127.9.96.67:40779)
0806 05:02:26.557825 (+ 975us) negotiation.cc:107] Waiting for socket to connect
0806 05:02:26.557860 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0806 05:02:26.559538 (+ 1678us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 05:02:26.560275 (+ 737us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 05:02:26.560282 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 05:02:26.560362 (+ 80us) client_negotiation.cc:189] Negotiated authn=SASL
0806 05:02:26.560917 (+ 555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 05:02:26.560933 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 05:02:26.562185 (+ 1252us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 05:02:26.562197 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 05:02:26.563830 (+ 1633us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 05:02:26.563845 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 05:02:26.564834 (+ 989us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 05:02:26.564842 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 05:02:26.565081 (+ 239us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:02:26.565813 (+ 732us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 05:02:26.565816 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0806 05:02:26.565857 (+ 41us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 05:02:26.565936 (+ 79us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 05:02:26.566333 (+ 397us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 05:02:26.566346 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 05:02:26.566356 (+ 10us) client_negotiation.cc:769] Sending connection context
0806 05:02:26.566552 (+ 196us) client_negotiation.cc:240] Negotiation successful
0806 05:02:26.566986 (+ 434us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":541,"thread_start_us":358,"threads_started":1}
I20260806 05:02:26.569455 6127 heartbeater.cc:347] Connected to a master server at 127.9.96.126:41191
I20260806 05:02:26.569756 6127 heartbeater.cc:464] Registering TS with master...
I20260806 05:02:26.570465 6127 heartbeater.cc:511] Master 127.9.96.126:41191 requested a full tablet report, sending...
I20260806 05:02:26.571939 5472 ts_manager.cc:194] Registered new tserver with Master: e18e0d8cca2946beb025190c17a0312f (127.9.96.67:45113)
I20260806 05:02:26.572677 5472 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.9.96.67:40779
I20260806 05:02:26.577181 9601 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 05:02:26.587760 5748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:02:26.581396 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57168 (local address 127.9.96.126:41191)
0806 05:02:26.581593 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0806 05:02:26.581597 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 05:02:26.582080 (+ 483us) server_negotiation.cc:407] Connection header received
0806 05:02:26.582195 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 05:02:26.582199 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 05:02:26.582279 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 05:02:26.582379 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0806 05:02:26.582914 (+ 535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:02:26.583729 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:02:26.585159 (+ 1430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:02:26.585556 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:02:26.585654 (+ 98us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:02:26.586962 (+ 1308us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 05:02:26.586971 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 05:02:26.586976 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 05:02:26.586999 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0806 05:02:26.587046 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 05:02:26.587060 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 05:02:26.587185 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0806 05:02:26.587488 (+ 303us) server_negotiation.cc:299] Negotiation successful
0806 05:02:26.587533 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260806 05:02:26.591877 5472 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:57168:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 05:02:26.598757 5472 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 05:02:26.621948 6144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:02:26.610333 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.67:45113 (local address 127.0.0.1:53842)
0806 05:02:26.611343 (+ 1010us) negotiation.cc:107] Waiting for socket to connect
0806 05:02:26.611409 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0806 05:02:26.611598 (+ 189us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 05:02:26.613930 (+ 2332us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 05:02:26.613937 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 05:02:26.613979 (+ 42us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 05:02:26.614410 (+ 431us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 05:02:26.614424 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 05:02:26.616730 (+ 2306us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 05:02:26.616735 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 05:02:26.619001 (+ 2266us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 05:02:26.619017 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 05:02:26.620971 (+ 1954us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 05:02:26.620977 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 05:02:26.621173 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:02:26.621247 (+ 74us) client_negotiation.cc:769] Sending connection context
0806 05:02:26.621377 (+ 130us) client_negotiation.cc:240] Negotiation successful
0806 05:02:26.621572 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":717,"mutex_wait_us":1,"thread_start_us":569,"threads_started":1}
I20260806 05:02:26.623888 6146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:02:26.610637 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53842 (local address 127.9.96.67:45113)
0806 05:02:26.613287 (+ 2650us) server_negotiation.cc:206] Beginning negotiation
0806 05:02:26.613296 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 05:02:26.613334 (+ 38us) server_negotiation.cc:407] Connection header received
0806 05:02:26.613434 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 05:02:26.613443 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 05:02:26.613688 (+ 245us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 05:02:26.614632 (+ 944us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 05:02:26.614932 (+ 300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:02:26.616563 (+ 1631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:02:26.619249 (+ 2686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:02:26.620724 (+ 1475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:02:26.622527 (+ 1803us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:02:26.622808 (+ 281us) server_negotiation.cc:1035] Waiting for connection context
0806 05:02:26.622879 (+ 71us) server_negotiation.cc:299] Negotiation successful
0806 05:02:26.623045 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2461,"thread_start_us":2458,"threads_started":1}
I20260806 05:02:26.625520 6145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:02:26.610628 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.66:39247 (local address 127.0.0.1:59266)
0806 05:02:26.613024 (+ 2396us) negotiation.cc:107] Waiting for socket to connect
0806 05:02:26.613044 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0806 05:02:26.613463 (+ 419us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 05:02:26.614327 (+ 864us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 05:02:26.614342 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 05:02:26.614375 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 05:02:26.614777 (+ 402us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 05:02:26.614790 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 05:02:26.617948 (+ 3158us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 05:02:26.617953 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 05:02:26.621303 (+ 3350us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 05:02:26.621315 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 05:02:26.624734 (+ 3419us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 05:02:26.624740 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 05:02:26.624935 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:02:26.624968 (+ 33us) client_negotiation.cc:769] Sending connection context
0806 05:02:26.625067 (+ 99us) client_negotiation.cc:240] Negotiation successful
0806 05:02:26.625212 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2255,"thread_start_us":1978,"threads_started":1}
I20260806 05:02:26.625640 6147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:02:26.611491 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59266 (local address 127.9.96.66:39247)
0806 05:02:26.612414 (+ 923us) server_negotiation.cc:206] Beginning negotiation
0806 05:02:26.612426 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 05:02:26.613354 (+ 928us) server_negotiation.cc:407] Connection header received
0806 05:02:26.613641 (+ 287us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 05:02:26.613650 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 05:02:26.613883 (+ 233us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 05:02:26.614041 (+ 158us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 05:02:26.614987 (+ 946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:02:26.617754 (+ 2767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:02:26.621507 (+ 3753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:02:26.623102 (+ 1595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:02:26.624356 (+ 1254us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:02:26.624703 (+ 347us) server_negotiation.cc:1035] Waiting for connection context
0806 05:02:26.625136 (+ 433us) server_negotiation.cc:299] Negotiation successful
0806 05:02:26.625301 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":732,"thread_start_us":618,"threads_started":1}
I20260806 05:02:26.628391 6038 tablet_service.cc:1467] Processing CreateTablet for tablet df5d6e069be04bc29084aadcfd830177 (DEFAULT_TABLE table=test-table [id=225c001faa5a4af8b46e0f12fcacd115]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 05:02:26.629354 6144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:02:26.611915 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.65:41859 (local address 127.0.0.1:41110)
0806 05:02:26.622200 (+ 10285us) negotiation.cc:107] Waiting for socket to connect
0806 05:02:26.622218 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 05:02:26.622323 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 05:02:26.623535 (+ 1212us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 05:02:26.623540 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 05:02:26.623567 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 05:02:26.623733 (+ 166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 05:02:26.623746 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 05:02:26.625840 (+ 2094us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 05:02:26.625846 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 05:02:26.627275 (+ 1429us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 05:02:26.627296 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 05:02:26.628595 (+ 1299us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 05:02:26.628601 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 05:02:26.628812 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:02:26.628845 (+ 33us) client_negotiation.cc:769] Sending connection context
0806 05:02:26.628909 (+ 64us) client_negotiation.cc:240] Negotiation successful
0806 05:02:26.629061 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10182,"thread_start_us":12255,"threads_started":1}
I20260806 05:02:26.629117 5849 tablet_service.cc:1467] Processing CreateTablet for tablet df5d6e069be04bc29084aadcfd830177 (DEFAULT_TABLE table=test-table [id=225c001faa5a4af8b46e0f12fcacd115]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 05:02:26.630165 6038 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet df5d6e069be04bc29084aadcfd830177. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 05:02:26.630493 6150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:02:26.612267 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41110 (local address 127.9.96.65:41859)
0806 05:02:26.622956 (+ 10689us) server_negotiation.cc:206] Beginning negotiation
0806 05:02:26.622966 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 05:02:26.622995 (+ 29us) server_negotiation.cc:407] Connection header received
0806 05:02:26.623080 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 05:02:26.623087 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 05:02:26.623324 (+ 237us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 05:02:26.623539 (+ 215us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 05:02:26.624191 (+ 652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:02:26.625655 (+ 1464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:02:26.627479 (+ 1824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:02:26.628352 (+ 873us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:02:26.629601 (+ 1249us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:02:26.629937 (+ 336us) server_negotiation.cc:1035] Waiting for connection context
0806 05:02:26.629992 (+ 55us) server_negotiation.cc:299] Negotiation successful
0806 05:02:26.630178 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10503,"thread_start_us":11426,"threads_started":1}
I20260806 05:02:26.630537 5849 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet df5d6e069be04bc29084aadcfd830177. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 05:02:26.634011 5648 tablet_service.cc:1467] Processing CreateTablet for tablet df5d6e069be04bc29084aadcfd830177 (DEFAULT_TABLE table=test-table [id=225c001faa5a4af8b46e0f12fcacd115]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 05:02:26.635531 5648 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet df5d6e069be04bc29084aadcfd830177. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 05:02:26.637682 5746 heartbeater.cc:503] Master 127.9.96.126:41191 was elected leader, sending a full tablet report...
I20260806 05:02:26.644014 6151 tablet_bootstrap.cc:492] T df5d6e069be04bc29084aadcfd830177 P 4468f796ea43458986fb979632e3f125: Bootstrap starting.
I20260806 05:02:26.646112 6152 tablet_bootstrap.cc:492] T df5d6e069be04bc29084aadcfd830177 P e18e0d8cca2946beb025190c17a0312f: Bootstrap starting.
I20260806 05:02:26.648309 6151 tablet_bootstrap.cc:654] T df5d6e069be04bc29084aadcfd830177 P 4468f796ea43458986fb979632e3f125: Neither blocks nor log segments found. Creating new log.
I20260806 05:02:26.649264 6154 tablet_bootstrap.cc:492] T df5d6e069be04bc29084aadcfd830177 P 0278739c639545aaa978f098bde50717: Bootstrap starting.
I20260806 05:02:26.649734 6151 log.cc:824] T df5d6e069be04bc29084aadcfd830177 P 4468f796ea43458986fb979632e3f125: Log is configured to *not* fsync() on all Append() calls
I20260806 05:02:26.649976 6152 tablet_bootstrap.cc:654] T df5d6e069be04bc29084aadcfd830177 P e18e0d8cca2946beb025190c17a0312f: Neither blocks nor log segments found. Creating new log.
I20260806 05:02:26.651343 6152 log.cc:824] T df5d6e069be04bc29084aadcfd830177 P e18e0d8cca2946beb025190c17a0312f: Log is configured to *not* fsync() on all Append() calls
I20260806 05:02:26.652854 6154 tablet_bootstrap.cc:654] T df5d6e069be04bc29084aadcfd830177 P 0278739c639545aaa978f098bde50717: Neither blocks nor log segments found. Creating new log.
I20260806 05:02:26.653460 6151 tablet_bootstrap.cc:492] T df5d6e069be04bc29084aadcfd830177 P 4468f796ea43458986fb979632e3f125: No bootstrap required, opened a new log
I20260806 05:02:26.653800 6151 ts_tablet_manager.cc:1397] T df5d6e069be04bc29084aadcfd830177 P 4468f796ea43458986fb979632e3f125: Time spent bootstrapping tablet: real 0.010s user 0.004s sys 0.004s
I20260806 05:02:26.654054 6154 log.cc:824] T df5d6e069be04bc29084aadcfd830177 P 0278739c639545aaa978f098bde50717: Log is configured to *not* fsync() on all Append() calls
I20260806 05:02:26.654655 6152 tablet_bootstrap.cc:492] T df5d6e069be04bc29084aadcfd830177 P e18e0d8cca2946beb025190c17a0312f: No bootstrap required, opened a new log
I20260806 05:02:26.654963 6152 ts_tablet_manager.cc:1397] T df5d6e069be04bc29084aadcfd830177 P e18e0d8cca2946beb025190c17a0312f: Time spent bootstrapping tablet: real 0.009s user 0.004s sys 0.004s
I20260806 05:02:26.657882 6154 tablet_bootstrap.cc:492] T df5d6e069be04bc29084aadcfd830177 P 0278739c639545aaa978f098bde50717: No bootstrap required, opened a new log
I20260806 05:02:26.658136 6154 ts_tablet_manager.cc:1397] T df5d6e069be04bc29084aadcfd830177 P 0278739c639545aaa978f098bde50717: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260806 05:02:26.664091 6152 raft_consensus.cc:348] T df5d6e069be04bc29084aadcfd830177 P e18e0d8cca2946beb025190c17a0312f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e18e0d8cca2946beb025190c17a0312f" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 45113 } } peers { permanent_uuid: "4468f796ea43458986fb979632e3f125" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 39247 } } peers { permanent_uuid: "0278739c639545aaa978f098bde50717" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 41859 } }
I20260806 05:02:26.664533 6152 raft_consensus.cc:374] T df5d6e069be04bc29084aadcfd830177 P e18e0d8cca2946beb025190c17a0312f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 05:02:26.664244 6151 raft_consensus.cc:348] T df5d6e069be04bc29084aadcfd830177 P 4468f796ea43458986fb979632e3f125 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e18e0d8cca2946beb025190c17a0312f" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 45113 } } peers { permanent_uuid: "4468f796ea43458986fb979632e3f125" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 39247 } } peers { permanent_uuid: "0278739c639545aaa978f098bde50717" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 41859 } }
I20260806 05:02:26.664677 6152 raft_consensus.cc:729] T df5d6e069be04bc29084aadcfd830177 P e18e0d8cca2946beb025190c17a0312f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e18e0d8cca2946beb025190c17a0312f, State: Initialized, Role: FOLLOWER
I20260806 05:02:26.664706 6151 raft_consensus.cc:374] T df5d6e069be04bc29084aadcfd830177 P 4468f796ea43458986fb979632e3f125 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 05:02:26.664816 6151 raft_consensus.cc:729] T df5d6e069be04bc29084aadcfd830177 P 4468f796ea43458986fb979632e3f125 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4468f796ea43458986fb979632e3f125, State: Initialized, Role: FOLLOWER
I20260806 05:02:26.665402 6152 consensus_queue.cc:260] T df5d6e069be04bc29084aadcfd830177 P e18e0d8cca2946beb025190c17a0312f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e18e0d8cca2946beb025190c17a0312f" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 45113 } } peers { permanent_uuid: "4468f796ea43458986fb979632e3f125" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 39247 } } peers { permanent_uuid: "0278739c639545aaa978f098bde50717" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 41859 } }
I20260806 05:02:26.665414 6151 consensus_queue.cc:260] T df5d6e069be04bc29084aadcfd830177 P 4468f796ea43458986fb979632e3f125 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e18e0d8cca2946beb025190c17a0312f" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 45113 } } peers { permanent_uuid: "4468f796ea43458986fb979632e3f125" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 39247 } } peers { permanent_uuid: "0278739c639545aaa978f098bde50717" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 41859 } }
I20260806 05:02:26.666818 5936 heartbeater.cc:503] Master 127.9.96.126:41191 was elected leader, sending a full tablet report...
I20260806 05:02:26.667269 6151 ts_tablet_manager.cc:1428] T df5d6e069be04bc29084aadcfd830177 P 4468f796ea43458986fb979632e3f125: Time spent starting tablet: real 0.013s user 0.009s sys 0.004s
I20260806 05:02:26.668184 6154 raft_consensus.cc:348] T df5d6e069be04bc29084aadcfd830177 P 0278739c639545aaa978f098bde50717 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e18e0d8cca2946beb025190c17a0312f" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 45113 } } peers { permanent_uuid: "4468f796ea43458986fb979632e3f125" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 39247 } } peers { permanent_uuid: "0278739c639545aaa978f098bde50717" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 41859 } }
I20260806 05:02:26.668546 6154 raft_consensus.cc:374] T df5d6e069be04bc29084aadcfd830177 P 0278739c639545aaa978f098bde50717 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 05:02:26.668668 6154 raft_consensus.cc:729] T df5d6e069be04bc29084aadcfd830177 P 0278739c639545aaa978f098bde50717 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0278739c639545aaa978f098bde50717, State: Initialized, Role: FOLLOWER
I20260806 05:02:26.669351 6154 consensus_queue.cc:260] T df5d6e069be04bc29084aadcfd830177 P 0278739c639545aaa978f098bde50717 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e18e0d8cca2946beb025190c17a0312f" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 45113 } } peers { permanent_uuid: "4468f796ea43458986fb979632e3f125" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 39247 } } peers { permanent_uuid: "0278739c639545aaa978f098bde50717" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 41859 } }
I20260806 05:02:26.669517 6127 heartbeater.cc:503] Master 127.9.96.126:41191 was elected leader, sending a full tablet report...
I20260806 05:02:26.670238 6152 ts_tablet_manager.cc:1428] T df5d6e069be04bc29084aadcfd830177 P e18e0d8cca2946beb025190c17a0312f: Time spent starting tablet: real 0.015s user 0.009s sys 0.005s
I20260806 05:02:26.673045 6154 ts_tablet_manager.cc:1428] T df5d6e069be04bc29084aadcfd830177 P 0278739c639545aaa978f098bde50717: Time spent starting tablet: real 0.015s user 0.013s sys 0.003s
I20260806 05:02:26.719650 6161 raft_consensus.cc:482] T df5d6e069be04bc29084aadcfd830177 P 0278739c639545aaa978f098bde50717 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 05:02:26.720003 6161 raft_consensus.cc:504] T df5d6e069be04bc29084aadcfd830177 P 0278739c639545aaa978f098bde50717 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e18e0d8cca2946beb025190c17a0312f" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 45113 } } peers { permanent_uuid: "4468f796ea43458986fb979632e3f125" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 39247 } } peers { permanent_uuid: "0278739c639545aaa978f098bde50717" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 41859 } }
I20260806 05:02:26.721753 6161 leader_election.cc:288] T df5d6e069be04bc29084aadcfd830177 P 0278739c639545aaa978f098bde50717 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e18e0d8cca2946beb025190c17a0312f (127.9.96.67:45113), 4468f796ea43458986fb979632e3f125 (127.9.96.66:39247)
I20260806 05:02:26.728767 6147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:02:26.722389 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.96.65:51969 (local address 127.9.96.66:39247)
0806 05:02:26.722670 (+ 281us) server_negotiation.cc:206] Beginning negotiation
0806 05:02:26.722679 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 05:02:26.723097 (+ 418us) server_negotiation.cc:407] Connection header received
0806 05:02:26.723331 (+ 234us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 05:02:26.723341 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 05:02:26.723476 (+ 135us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 05:02:26.723576 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 05:02:26.724110 (+ 534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:02:26.724920 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:02:26.726898 (+ 1978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:02:26.727796 (+ 898us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:02:26.727894 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:02:26.727971 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0806 05:02:26.728363 (+ 392us) server_negotiation.cc:299] Negotiation successful
0806 05:02:26.728499 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
I20260806 05:02:26.728803 6146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:02:26.722143 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.96.65:41387 (local address 127.9.96.67:45113)
0806 05:02:26.722359 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0806 05:02:26.722367 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 05:02:26.723398 (+ 1031us) server_negotiation.cc:407] Connection header received
0806 05:02:26.723620 (+ 222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 05:02:26.723624 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 05:02:26.723682 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 05:02:26.723789 (+ 107us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 05:02:26.724453 (+ 664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:02:26.725289 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:02:26.727109 (+ 1820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:02:26.727936 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:02:26.728068 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:02:26.728173 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0806 05:02:26.728430 (+ 257us) server_negotiation.cc:299] Negotiation successful
0806 05:02:26.728555 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260806 05:02:26.728924 6165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:02:26.722182 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.67:45113 (local address 127.9.96.65:41387)
0806 05:02:26.723255 (+ 1073us) negotiation.cc:107] Waiting for socket to connect
0806 05:02:26.723269 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 05:02:26.723441 (+ 172us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 05:02:26.723813 (+ 372us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 05:02:26.723817 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 05:02:26.723842 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 05:02:26.724263 (+ 421us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 05:02:26.724272 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 05:02:26.725424 (+ 1152us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 05:02:26.725427 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 05:02:26.726917 (+ 1490us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 05:02:26.726931 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 05:02:26.728121 (+ 1190us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 05:02:26.728126 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 05:02:26.728240 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:02:26.728262 (+ 22us) client_negotiation.cc:769] Sending connection context
0806 05:02:26.728376 (+ 114us) client_negotiation.cc:240] Negotiation successful
0806 05:02:26.728512 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":956,"mutex_wait_us":30,"thread_start_us":683,"threads_started":1}
I20260806 05:02:26.729391 6163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:02:26.722031 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.66:39247 (local address 127.9.96.65:51969)
0806 05:02:26.722968 (+ 937us) negotiation.cc:107] Waiting for socket to connect
0806 05:02:26.722985 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 05:02:26.723175 (+ 190us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 05:02:26.723593 (+ 418us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 05:02:26.723597 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 05:02:26.723622 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 05:02:26.723964 (+ 342us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 05:02:26.723974 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 05:02:26.725074 (+ 1100us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 05:02:26.725080 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 05:02:26.726692 (+ 1612us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 05:02:26.726704 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 05:02:26.727993 (+ 1289us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 05:02:26.728001 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 05:02:26.728179 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:02:26.728218 (+ 39us) client_negotiation.cc:769] Sending connection context
0806 05:02:26.728302 (+ 84us) client_negotiation.cc:240] Negotiation successful
0806 05:02:26.728476 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":814,"mutex_wait_us":32,"thread_start_us":616,"threads_started":1}
I20260806 05:02:26.729955 6068 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "df5d6e069be04bc29084aadcfd830177" candidate_uuid: "0278739c639545aaa978f098bde50717" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e18e0d8cca2946beb025190c17a0312f" is_pre_election: true
I20260806 05:02:26.730504 6068 raft_consensus.cc:2454] T df5d6e069be04bc29084aadcfd830177 P e18e0d8cca2946beb025190c17a0312f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0278739c639545aaa978f098bde50717 in term 0.
I20260806 05:02:26.730393 5878 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "df5d6e069be04bc29084aadcfd830177" candidate_uuid: "0278739c639545aaa978f098bde50717" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4468f796ea43458986fb979632e3f125" is_pre_election: true
I20260806 05:02:26.730922 5878 raft_consensus.cc:2454] T df5d6e069be04bc29084aadcfd830177 P 4468f796ea43458986fb979632e3f125 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0278739c639545aaa978f098bde50717 in term 0.
I20260806 05:02:26.731248 5580 leader_election.cc:302] T df5d6e069be04bc29084aadcfd830177 P 0278739c639545aaa978f098bde50717 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0278739c639545aaa978f098bde50717, e18e0d8cca2946beb025190c17a0312f; no voters:
I20260806 05:02:26.731698 6161 raft_consensus.cc:2781] T df5d6e069be04bc29084aadcfd830177 P 0278739c639545aaa978f098bde50717 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 05:02:26.731866 6161 raft_consensus.cc:482] T df5d6e069be04bc29084aadcfd830177 P 0278739c639545aaa978f098bde50717 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 05:02:26.731994 6161 raft_consensus.cc:3037] T df5d6e069be04bc29084aadcfd830177 P 0278739c639545aaa978f098bde50717 [term 0 FOLLOWER]: Advancing to term 1
I20260806 05:02:26.734176 6161 raft_consensus.cc:504] T df5d6e069be04bc29084aadcfd830177 P 0278739c639545aaa978f098bde50717 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e18e0d8cca2946beb025190c17a0312f" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 45113 } } peers { permanent_uuid: "4468f796ea43458986fb979632e3f125" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 39247 } } peers { permanent_uuid: "0278739c639545aaa978f098bde50717" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 41859 } }
I20260806 05:02:26.734819 6161 leader_election.cc:288] T df5d6e069be04bc29084aadcfd830177 P 0278739c639545aaa978f098bde50717 [CANDIDATE]: Term 1 election: Requested vote from peers e18e0d8cca2946beb025190c17a0312f (127.9.96.67:45113), 4468f796ea43458986fb979632e3f125 (127.9.96.66:39247)
I20260806 05:02:26.735311 6068 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "df5d6e069be04bc29084aadcfd830177" candidate_uuid: "0278739c639545aaa978f098bde50717" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e18e0d8cca2946beb025190c17a0312f"
I20260806 05:02:26.735349 5878 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "df5d6e069be04bc29084aadcfd830177" candidate_uuid: "0278739c639545aaa978f098bde50717" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4468f796ea43458986fb979632e3f125"
I20260806 05:02:26.735575 6068 raft_consensus.cc:3037] T df5d6e069be04bc29084aadcfd830177 P e18e0d8cca2946beb025190c17a0312f [term 0 FOLLOWER]: Advancing to term 1
I20260806 05:02:26.735584 5878 raft_consensus.cc:3037] T df5d6e069be04bc29084aadcfd830177 P 4468f796ea43458986fb979632e3f125 [term 0 FOLLOWER]: Advancing to term 1
I20260806 05:02:26.737884 6068 raft_consensus.cc:2454] T df5d6e069be04bc29084aadcfd830177 P e18e0d8cca2946beb025190c17a0312f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0278739c639545aaa978f098bde50717 in term 1.
I20260806 05:02:26.737982 5878 raft_consensus.cc:2454] T df5d6e069be04bc29084aadcfd830177 P 4468f796ea43458986fb979632e3f125 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0278739c639545aaa978f098bde50717 in term 1.
I20260806 05:02:26.738314 5580 leader_election.cc:302] T df5d6e069be04bc29084aadcfd830177 P 0278739c639545aaa978f098bde50717 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0278739c639545aaa978f098bde50717, e18e0d8cca2946beb025190c17a0312f; no voters:
I20260806 05:02:26.738590 6161 raft_consensus.cc:2781] T df5d6e069be04bc29084aadcfd830177 P 0278739c639545aaa978f098bde50717 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 05:02:26.739035 6161 raft_consensus.cc:686] T df5d6e069be04bc29084aadcfd830177 P 0278739c639545aaa978f098bde50717 [term 1 LEADER]: Becoming Leader. State: Replica: 0278739c639545aaa978f098bde50717, State: Running, Role: LEADER
I20260806 05:02:26.739547 6161 consensus_queue.cc:237] T df5d6e069be04bc29084aadcfd830177 P 0278739c639545aaa978f098bde50717 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e18e0d8cca2946beb025190c17a0312f" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 45113 } } peers { permanent_uuid: "4468f796ea43458986fb979632e3f125" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 39247 } } peers { permanent_uuid: "0278739c639545aaa978f098bde50717" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 41859 } }
I20260806 05:02:26.744272 5462 catalog_manager.cc:5495] T df5d6e069be04bc29084aadcfd830177 P 0278739c639545aaa978f098bde50717 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0278739c639545aaa978f098bde50717 (127.9.96.65). New cstate: current_term: 1 leader_uuid: "0278739c639545aaa978f098bde50717" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e18e0d8cca2946beb025190c17a0312f" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 45113 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4468f796ea43458986fb979632e3f125" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 39247 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0278739c639545aaa978f098bde50717" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 41859 } health_report { overall_health: HEALTHY } } }
I20260806 05:02:26.778702 6150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:02:26.772845 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41124 (local address 127.9.96.65:41859)
0806 05:02:26.773058 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0806 05:02:26.773062 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 05:02:26.773251 (+ 189us) server_negotiation.cc:407] Connection header received
0806 05:02:26.773429 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 05:02:26.773433 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 05:02:26.773512 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 05:02:26.773644 (+ 132us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 05:02:26.774027 (+ 383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:02:26.774748 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:02:26.776393 (+ 1645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:02:26.776814 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:02:26.776912 (+ 98us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:02:26.777557 (+ 645us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 05:02:26.778045 (+ 488us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 05:02:26.778229 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0806 05:02:26.778453 (+ 224us) server_negotiation.cc:299] Negotiation successful
0806 05:02:26.778492 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260806 05:02:26.787602 5878 raft_consensus.cc:1261] T df5d6e069be04bc29084aadcfd830177 P 4468f796ea43458986fb979632e3f125 [term 1 FOLLOWER]: Refusing update from remote peer 0278739c639545aaa978f098bde50717: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 05:02:26.787638 6068 raft_consensus.cc:1261] T df5d6e069be04bc29084aadcfd830177 P e18e0d8cca2946beb025190c17a0312f [term 1 FOLLOWER]: Refusing update from remote peer 0278739c639545aaa978f098bde50717: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 05:02:26.788713 6166 consensus_queue.cc:1035] T df5d6e069be04bc29084aadcfd830177 P 0278739c639545aaa978f098bde50717 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e18e0d8cca2946beb025190c17a0312f" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 45113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 05:02:26.789098 6161 consensus_queue.cc:1035] T df5d6e069be04bc29084aadcfd830177 P 0278739c639545aaa978f098bde50717 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4468f796ea43458986fb979632e3f125" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 39247 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 05:02:26.805248 6173 mvcc.cc:205] Tried to move back new op lower bound from 7315425471630610432 to 7315425471448883200. Current Snapshot: MvccSnapshot[applied={T|T < 7315425471630610432}]
I20260806 05:02:26.818010 5465 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:57168:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 05:02:26.818274 5465 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:57168:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 05:02:26.822428 5465 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 24d4fe668b6748b7a3016815025a8830: Sending DeleteTablet for 3 replicas of tablet df5d6e069be04bc29084aadcfd830177
I20260806 05:02:26.823745 6038 tablet_service.cc:1514] Processing DeleteTablet for tablet df5d6e069be04bc29084aadcfd830177 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 05:02:26 UTC) from {username='slave'} at 127.0.0.1:53842
I20260806 05:02:26.823745 5849 tablet_service.cc:1514] Processing DeleteTablet for tablet df5d6e069be04bc29084aadcfd830177 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 05:02:26 UTC) from {username='slave'} at 127.0.0.1:59266
I20260806 05:02:26.824124 5648 tablet_service.cc:1514] Processing DeleteTablet for tablet df5d6e069be04bc29084aadcfd830177 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 05:02:26 UTC) from {username='slave'} at 127.0.0.1:41110
I20260806 05:02:26.825899 9601 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu pid 5559
I20260806 05:02:26.836634 6189 tablet_replica.cc:330] T df5d6e069be04bc29084aadcfd830177 P 4468f796ea43458986fb979632e3f125: stopping tablet replica
I20260806 05:02:26.837385 6189 raft_consensus.cc:2229] T df5d6e069be04bc29084aadcfd830177 P 4468f796ea43458986fb979632e3f125 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 05:02:26.838145 6189 raft_consensus.cc:2258] T df5d6e069be04bc29084aadcfd830177 P 4468f796ea43458986fb979632e3f125 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 05:02:26.840878 6189 ts_tablet_manager.cc:1905] T df5d6e069be04bc29084aadcfd830177 P 4468f796ea43458986fb979632e3f125: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 05:02:26.841110 6187 tablet_replica.cc:330] T df5d6e069be04bc29084aadcfd830177 P e18e0d8cca2946beb025190c17a0312f: stopping tablet replica
I20260806 05:02:26.841571 6187 raft_consensus.cc:2229] T df5d6e069be04bc29084aadcfd830177 P e18e0d8cca2946beb025190c17a0312f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 05:02:26.843799 6187 raft_consensus.cc:2258] T df5d6e069be04bc29084aadcfd830177 P e18e0d8cca2946beb025190c17a0312f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 05:02:26.846099 6190 tablet_replica.cc:330] T df5d6e069be04bc29084aadcfd830177 P 0278739c639545aaa978f098bde50717: stopping tablet replica
I20260806 05:02:26.846508 6190 raft_consensus.cc:2229] T df5d6e069be04bc29084aadcfd830177 P 0278739c639545aaa978f098bde50717 [term 1 LEADER]: Raft consensus shutting down.
I20260806 05:02:26.847333 6190 raft_consensus.cc:2258] T df5d6e069be04bc29084aadcfd830177 P 0278739c639545aaa978f098bde50717 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 05:02:26.847635 6187 ts_tablet_manager.cc:1905] T df5d6e069be04bc29084aadcfd830177 P e18e0d8cca2946beb025190c17a0312f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 05:02:26.848910 6150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:02:26.827511 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41136 (local address 127.9.96.65:41859)
0806 05:02:26.827749 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0806 05:02:26.827759 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 05:02:26.839641 (+ 11882us) server_negotiation.cc:407] Connection header received
0806 05:02:26.839958 (+ 317us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 05:02:26.839963 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 05:02:26.840068 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 05:02:26.840175 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0806 05:02:26.840888 (+ 713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:02:26.842105 (+ 1217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:02:26.843984 (+ 1879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:02:26.844669 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:02:26.844765 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:02:26.846821 (+ 2056us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 05:02:26.846835 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 05:02:26.846854 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 05:02:26.847777 (+ 923us) server_negotiation.cc:922] calling sasl_server_start()
0806 05:02:26.847859 (+ 82us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 05:02:26.847890 (+ 31us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 05:02:26.847978 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0806 05:02:26.848339 (+ 361us) server_negotiation.cc:299] Negotiation successful
0806 05:02:26.848451 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260806 05:02:26.850538 5741 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 05:02:26.851791 6189 ts_tablet_manager.cc:1918] T df5d6e069be04bc29084aadcfd830177 P 4468f796ea43458986fb979632e3f125: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 05:02:26.851970 6189 log.cc:1196] T df5d6e069be04bc29084aadcfd830177 P 4468f796ea43458986fb979632e3f125: Deleting WAL directory at /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785992227050689-9601-0/minicluster-data/ts-1/wal/wals/df5d6e069be04bc29084aadcfd830177
I20260806 05:02:26.852530 6189 ts_tablet_manager.cc:1939] T df5d6e069be04bc29084aadcfd830177 P 4468f796ea43458986fb979632e3f125: Deleting consensus metadata
I20260806 05:02:26.854627 6187 ts_tablet_manager.cc:1918] T df5d6e069be04bc29084aadcfd830177 P e18e0d8cca2946beb025190c17a0312f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 05:02:26.854794 6187 log.cc:1196] T df5d6e069be04bc29084aadcfd830177 P e18e0d8cca2946beb025190c17a0312f: Deleting WAL directory at /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785992227050689-9601-0/minicluster-data/ts-2/wal/wals/df5d6e069be04bc29084aadcfd830177
I20260806 05:02:26.854914 5443 catalog_manager.cc:4841] TS 4468f796ea43458986fb979632e3f125 (127.9.96.66:39247): tablet df5d6e069be04bc29084aadcfd830177 (table test-table [id=225c001faa5a4af8b46e0f12fcacd115]) successfully deleted
I20260806 05:02:26.855393 6187 ts_tablet_manager.cc:1939] T df5d6e069be04bc29084aadcfd830177 P e18e0d8cca2946beb025190c17a0312f: Deleting consensus metadata
I20260806 05:02:26.857220 5444 catalog_manager.cc:4841] TS e18e0d8cca2946beb025190c17a0312f (127.9.96.67:45113): tablet df5d6e069be04bc29084aadcfd830177 (table test-table [id=225c001faa5a4af8b46e0f12fcacd115]) successfully deleted
I20260806 05:02:26.980824 9601 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu with pid 5559
W20260806 05:02:27.012705 5444 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.9.96.65:41859 (error 108)
I20260806 05:02:27.013291 9601 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu pid 5750
I20260806 05:02:27.014756 6144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:02:27.014158 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.65:41859 (local address 127.0.0.1:41150)
0806 05:02:27.014400 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0806 05:02:27.014546 (+ 146us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.96.65:41859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105}
W20260806 05:02:27.015058 5444 catalog_manager.cc:4584] TS 0278739c639545aaa978f098bde50717 (127.9.96.65:41859): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet df5d6e069be04bc29084aadcfd830177: Network error: Client connection negotiation failed: client connection to 127.9.96.65:41859: connect: Connection refused (error 111)
I20260806 05:02:27.020756 6147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:02:27.014161 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59268 (local address 127.9.96.66:39247)
0806 05:02:27.014352 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0806 05:02:27.014358 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 05:02:27.014381 (+ 23us) server_negotiation.cc:407] Connection header received
0806 05:02:27.014562 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 05:02:27.014568 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 05:02:27.014691 (+ 123us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 05:02:27.014851 (+ 160us) server_negotiation.cc:226] Negotiated authn=SASL
0806 05:02:27.015346 (+ 495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:02:27.016151 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:02:27.017509 (+ 1358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:02:27.017918 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:02:27.018022 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:02:27.019134 (+ 1112us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 05:02:27.019143 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 05:02:27.019156 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 05:02:27.019932 (+ 776us) server_negotiation.cc:922] calling sasl_server_start()
0806 05:02:27.019982 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 05:02:27.020003 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 05:02:27.020119 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0806 05:02:27.020271 (+ 152us) server_negotiation.cc:299] Negotiation successful
0806 05:02:27.020363 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260806 05:02:27.021411 5927 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 05:02:27.069067 6144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:02:27.068446 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.65:41859 (local address 127.0.0.1:41162)
0806 05:02:27.068751 (+ 305us) negotiation.cc:107] Waiting for socket to connect
0806 05:02:27.068889 (+ 138us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.96.65:41859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":143}
I20260806 05:02:27.070261 6144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:02:27.069840 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.65:41859 (local address 127.0.0.1:41164)
0806 05:02:27.070010 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0806 05:02:27.070095 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.96.65:41859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260806 05:02:27.110112 9601 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu with pid 5750
I20260806 05:02:27.130838 6144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:02:27.130388 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.65:41859 (local address 127.0.0.1:41166)
0806 05:02:27.130602 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0806 05:02:27.130688 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.96.65:41859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
I20260806 05:02:27.131973 6144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:02:27.131622 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.65:41859 (local address 127.0.0.1:41180)
0806 05:02:27.131781 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0806 05:02:27.131860 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.96.65:41859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260806 05:02:27.136770 9601 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu pid 5942
I20260806 05:02:27.143777 6146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:02:27.137470 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53852 (local address 127.9.96.67:45113)
0806 05:02:27.137670 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0806 05:02:27.137676 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 05:02:27.137696 (+ 20us) server_negotiation.cc:407] Connection header received
0806 05:02:27.137893 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 05:02:27.137905 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 05:02:27.137992 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 05:02:27.138122 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0806 05:02:27.138528 (+ 406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:02:27.139278 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:02:27.140651 (+ 1373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:02:27.141035 (+ 384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:02:27.141172 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:02:27.142306 (+ 1134us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 05:02:27.142315 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 05:02:27.142330 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 05:02:27.142896 (+ 566us) server_negotiation.cc:922] calling sasl_server_start()
0806 05:02:27.142942 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 05:02:27.142962 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 05:02:27.143098 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0806 05:02:27.143246 (+ 148us) server_negotiation.cc:299] Negotiation successful
0806 05:02:27.143353 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260806 05:02:27.144445 6120 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 05:02:27.212774 6144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:02:27.212219 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.65:41859 (local address 127.0.0.1:41188)
0806 05:02:27.212466 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0806 05:02:27.212582 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.96.65:41859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":84}
I20260806 05:02:27.214057 6144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:02:27.213533 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.65:41859 (local address 127.0.0.1:41196)
0806 05:02:27.213770 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0806 05:02:27.213890 (+ 120us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.96.65:41859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125}
I20260806 05:02:27.233557 9601 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu with pid 5942
I20260806 05:02:27.259236 9601 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu pid 5425
I20260806 05:02:27.260394 5521 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 05:02:27.368469 9601 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu with pid 5425
2026-08-06T05:02:27Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (2840 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785992227050689-9601-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 05:02:27 dist-test-slave-x4qh krb5kdc[6200](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 05:02:27 dist-test-slave-x4qh krb5kdc[6200](info): set up 2 sockets
Aug 06 05:02:27 dist-test-slave-x4qh krb5kdc[6200](info): commencing operation
krb5kdc: starting...
W20260806 05:02:28.700498 9601 mini_kdc.cc:121] Time spent starting KDC: real 1.280s user 0.001s sys 0.043s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785992227050689-9601-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785992227050689-9601-0/krb5kdc/test-user.keytab.
Aug 06 05:02:28 dist-test-slave-x4qh krb5kdc[6200](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992548, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T05:02:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T05:02:28Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.96.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.96.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.96.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.126.keytab.
Entry for principal kudu/127.9.96.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.126.keytab.
I20260806 05:02:28.988615 9601 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
/tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785992227050689-9601-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785992227050689-9601-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785992227050689-9601-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785992227050689-9601-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.96.126:38143
--webserver_interface=127.9.96.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.126.keytab
--principal=kudu/127.9.96.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.96.84:44643
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.96.126:38143
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785992227050689-9601-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785992227050689-9601-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785992227050689-9601-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 05:02:29.325605 6219 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 05:02:29.325893 6219 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 05:02:29.326059 6219 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 05:02:29.333214 6219 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 05:02:29.333317 6219 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 05:02:29.333437 6219 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 05:02:29.333534 6219 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 05:02:29.333611 6219 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 05:02:29.343235 6219 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.96.84:44643
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785992227050689-9601-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785992227050689-9601-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.96.126:38143
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.126.keytab
--principal=kudu/127.9.96.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.96.126:38143
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785992227050689-9601-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.96.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785992227050689-9601-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 03:56:12 UTC on ccycloud.quasar-guijbm.root.comops.site
build id 4569618
ASAN enabled
I20260806 05:02:29.344568 6219 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 05:02:29.345842 6219 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 05:02:29.357931 6224 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 05:02:29.358033 6228 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 05:02:29.357935 6225 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 05:02:29.359107 6219 server_base.cc:1034] running on GCE node
I20260806 05:02:29.359957 6219 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 05:02:29.361340 6219 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 05:02:29.362545 6219 hybrid_clock.cc:648] HybridClock initialized: now 1785992549362497 us; error 34 us; skew 500 ppm
I20260806 05:02:29.362910 6219 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 05:02:29 dist-test-slave-x4qh krb5kdc[6200](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992549, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.96.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 05:02:29.370518 6219 init.cc:377] Logged in from keytab as kudu/127.9.96.126@KRBTEST.COM (short username kudu)
I20260806 05:02:29.372231 6219 webserver.cc:460] Webserver started at http://127.9.96.126:40967/ using document root <none> and password file <none>
I20260806 05:02:29.373060 6219 fs_manager.cc:359] Metadata directory not provided
I20260806 05:02:29.373243 6219 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 05:02:29.373584 6219 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 05:02:29.376735 6219 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785992227050689-9601-0/minicluster-data/master-0/data/instance:
uuid: "9e006065b18b49318c102b46e1ced923"
format_stamp: "Formatted at 2026-08-06 05:02:29 on dist-test-slave-x4qh"
server_key: "24e365bac1bc940ef0709aa7eba7c6a8"
server_key_iv: "1bdfdce581d483ca8d2b6baaf0a5002e"
server_key_version: "encryptionkey@0"
I20260806 05:02:29.377539 6219 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785992227050689-9601-0/minicluster-data/master-0/wal/instance:
uuid: "9e006065b18b49318c102b46e1ced923"
format_stamp: "Formatted at 2026-08-06 05:02:29 on dist-test-slave-x4qh"
server_key: "24e365bac1bc940ef0709aa7eba7c6a8"
server_key_iv: "1bdfdce581d483ca8d2b6baaf0a5002e"
server_key_version: "encryptionkey@0"
I20260806 05:02:29.382985 6219 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260806 05:02:29.386690 6238 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 05:02:29.388188 6219 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 05:02:29.388391 6219 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785992227050689-9601-0/minicluster-data/master-0/data,/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785992227050689-9601-0/minicluster-data/master-0/wal
uuid: "9e006065b18b49318c102b46e1ced923"
format_stamp: "Formatted at 2026-08-06 05:02:29 on dist-test-slave-x4qh"
server_key: "24e365bac1bc940ef0709aa7eba7c6a8"
server_key_iv: "1bdfdce581d483ca8d2b6baaf0a5002e"
server_key_version: "encryptionkey@0"
I20260806 05:02:29.388589 6219 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785992227050689-9601-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785992227050689-9601-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785992227050689-9601-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 05:02:29.404587 6219 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 05:02:29.408558 6219 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 05:02:29.408852 6219 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 05:02:29.427929 6219 rpc_server.cc:307] RPC server started. Bound to: 127.9.96.126:38143
I20260806 05:02:29.427969 6317 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.96.126:38143 every 8 connection(s)
I20260806 05:02:29.429278 6219 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785992227050689-9601-0/minicluster-data/master-0/data/info.pb
I20260806 05:02:29.432991 6318 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 05:02:29.437608 9601 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu as pid 6219
I20260806 05:02:29.437777 9601 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785992227050689-9601-0/minicluster-data/master-0/wal/instance
I20260806 05:02:29.438238 9601 external_mini_cluster.cc:1428] Setting key 0ec94f90eb96be24da5ab08dc18dec82
I20260806 05:02:29.445370 6318 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9e006065b18b49318c102b46e1ced923: Bootstrap starting.
I20260806 05:02:29.449085 6318 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9e006065b18b49318c102b46e1ced923: Neither blocks nor log segments found. Creating new log.
I20260806 05:02:29.450201 6318 log.cc:824] T 00000000000000000000000000000000 P 9e006065b18b49318c102b46e1ced923: Log is configured to *not* fsync() on all Append() calls
I20260806 05:02:29.453169 6318 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9e006065b18b49318c102b46e1ced923: No bootstrap required, opened a new log
Aug 06 05:02:29 dist-test-slave-x4qh krb5kdc[6200](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992548, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.96.126@KRBTEST.COM
I20260806 05:02:29.462625 6318 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9e006065b18b49318c102b46e1ced923 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e006065b18b49318c102b46e1ced923" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 38143 } }
I20260806 05:02:29.463119 6318 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9e006065b18b49318c102b46e1ced923 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 05:02:29.463255 6318 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9e006065b18b49318c102b46e1ced923 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e006065b18b49318c102b46e1ced923, State: Initialized, Role: FOLLOWER
I20260806 05:02:29.463861 6318 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9e006065b18b49318c102b46e1ced923 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e006065b18b49318c102b46e1ced923" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 38143 } }
I20260806 05:02:29.464134 6318 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9e006065b18b49318c102b46e1ced923 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 05:02:29.464308 6318 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9e006065b18b49318c102b46e1ced923 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 05:02:29.464324 6325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:02:29.440204 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40404 (local address 127.9.96.126:38143)
0806 05:02:29.441074 (+ 870us) server_negotiation.cc:206] Beginning negotiation
0806 05:02:29.441091 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0806 05:02:29.441144 (+ 53us) server_negotiation.cc:407] Connection header received
0806 05:02:29.443748 (+ 2604us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 05:02:29.443779 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 05:02:29.444266 (+ 487us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 05:02:29.444718 (+ 452us) server_negotiation.cc:226] Negotiated authn=SASL
0806 05:02:29.446365 (+ 1647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:02:29.448457 (+ 2092us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:02:29.449876 (+ 1419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:02:29.450993 (+ 1117us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:02:29.451135 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:02:29.455808 (+ 4673us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 05:02:29.455815 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 05:02:29.455828 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 05:02:29.455870 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0806 05:02:29.459563 (+ 3693us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 05:02:29.460510 (+ 947us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 05:02:29.460522 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 05:02:29.460528 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 05:02:29.460739 (+ 211us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 05:02:29.461191 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 05:02:29.461195 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 05:02:29.461197 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 05:02:29.461795 (+ 598us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 05:02:29.462029 (+ 234us) server_negotiation.cc:1035] Waiting for connection context
0806 05:02:29.462547 (+ 518us) server_negotiation.cc:299] Negotiation successful
0806 05:02:29.462951 (+ 404us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":599,"thread_start_us":356,"threads_started":1}
I20260806 05:02:29.464530 6318 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9e006065b18b49318c102b46e1ced923 [term 0 FOLLOWER]: Advancing to term 1
I20260806 05:02:29.468184 6318 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9e006065b18b49318c102b46e1ced923 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e006065b18b49318c102b46e1ced923" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 38143 } }
I20260806 05:02:29.468743 6318 leader_election.cc:302] T 00000000000000000000000000000000 P 9e006065b18b49318c102b46e1ced923 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9e006065b18b49318c102b46e1ced923; no voters:
I20260806 05:02:29.469352 6318 leader_election.cc:288] T 00000000000000000000000000000000 P 9e006065b18b49318c102b46e1ced923 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 05:02:29.469520 6328 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9e006065b18b49318c102b46e1ced923 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 05:02:29.470361 6328 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9e006065b18b49318c102b46e1ced923 [term 1 LEADER]: Becoming Leader. State: Replica: 9e006065b18b49318c102b46e1ced923, State: Running, Role: LEADER
I20260806 05:02:29.470904 6328 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9e006065b18b49318c102b46e1ced923 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e006065b18b49318c102b46e1ced923" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 38143 } }
I20260806 05:02:29.471498 6318 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9e006065b18b49318c102b46e1ced923 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 05:02:29.473680 6329 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9e006065b18b49318c102b46e1ced923 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9e006065b18b49318c102b46e1ced923" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e006065b18b49318c102b46e1ced923" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 38143 } } }
I20260806 05:02:29.474144 6329 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9e006065b18b49318c102b46e1ced923 [sys.catalog]: This master's current role is: LEADER
I20260806 05:02:29.475039 6330 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9e006065b18b49318c102b46e1ced923 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9e006065b18b49318c102b46e1ced923. Latest consensus state: current_term: 1 leader_uuid: "9e006065b18b49318c102b46e1ced923" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e006065b18b49318c102b46e1ced923" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 38143 } } }
I20260806 05:02:29.475348 6330 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9e006065b18b49318c102b46e1ced923 [sys.catalog]: This master's current role is: LEADER
I20260806 05:02:29.476015 6336 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 05:02:29.481698 6336 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 05:02:29.491544 6336 catalog_manager.cc:1347] Generated new cluster ID: cd5c5d8f647147a19801d5ba4384427b
I20260806 05:02:29.491659 6336 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 05:02:29.511786 6336 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 05:02:29.513631 6336 catalog_manager.cc:1504] Loading token signing keys...
I20260806 05:02:29.522161 6336 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9e006065b18b49318c102b46e1ced923: Generated new TSK 0
I20260806 05:02:29.523161 6336 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.9.96.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.96.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.96.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.65.keytab.
Entry for principal kudu/127.9.96.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.65.keytab.
I20260806 05:02:29.596254 9601 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
/tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785992227050689-9601-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785992227050689-9601-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785992227050689-9601-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785992227050689-9601-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.96.65:0
--local_ip_for_outbound_sockets=127.9.96.65
--webserver_interface=127.9.96.65
--webserver_port=0
--tserver_master_addrs=127.9.96.126:38143
--keytab_file=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.65.keytab
--principal=kudu/127.9.96.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.96.84:44643
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785992227050689-9601-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785992227050689-9601-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785992227050689-9601-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 05:02:29.933168 6360 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 05:02:29.933434 6360 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 05:02:29.933591 6360 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 05:02:29.941273 6360 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 05:02:29.941370 6360 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 05:02:29.941557 6360 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.96.65
I20260806 05:02:29.951122 6360 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.96.84:44643
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785992227050689-9601-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785992227050689-9601-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.65.keytab
--principal=kudu/127.9.96.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.96.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785992227050689-9601-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.96.65
--webserver_port=0
--tserver_master_addrs=127.9.96.126:38143
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.96.65
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785992227050689-9601-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 03:56:12 UTC on ccycloud.quasar-guijbm.root.comops.site
build id 4569618
ASAN enabled
I20260806 05:02:29.952383 6360 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 05:02:29.953660 6360 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 05:02:29.967288 6367 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 05:02:29.967355 6370 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 05:02:29.967360 6368 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 05:02:29.968214 6360 server_base.cc:1034] running on GCE node
I20260806 05:02:29.968779 6360 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 05:02:29.969636 6360 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 05:02:29.970887 6360 hybrid_clock.cc:648] HybridClock initialized: now 1785992549970827 us; error 46 us; skew 500 ppm
I20260806 05:02:29.971226 6360 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 05:02:29 dist-test-slave-x4qh krb5kdc[6200](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992549, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.96.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 05:02:29.974375 6360 init.cc:377] Logged in from keytab as kudu/127.9.96.65@KRBTEST.COM (short username kudu)
I20260806 05:02:29.976089 6360 webserver.cc:460] Webserver started at http://127.9.96.65:39543/ using document root <none> and password file <none>
I20260806 05:02:29.976758 6360 fs_manager.cc:359] Metadata directory not provided
I20260806 05:02:29.976902 6360 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 05:02:29.977233 6360 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 05:02:29.980140 6360 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785992227050689-9601-0/minicluster-data/ts-0/data/instance:
uuid: "87fbe18fcd9b43de909b2ac042447d30"
format_stamp: "Formatted at 2026-08-06 05:02:29 on dist-test-slave-x4qh"
server_key: "3679d4564586d32fcd913222ebc1e68c"
server_key_iv: "7df72004d09007a404f4a71ca05175d2"
server_key_version: "encryptionkey@0"
I20260806 05:02:29.980996 6360 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785992227050689-9601-0/minicluster-data/ts-0/wal/instance:
uuid: "87fbe18fcd9b43de909b2ac042447d30"
format_stamp: "Formatted at 2026-08-06 05:02:29 on dist-test-slave-x4qh"
server_key: "3679d4564586d32fcd913222ebc1e68c"
server_key_iv: "7df72004d09007a404f4a71ca05175d2"
server_key_version: "encryptionkey@0"
I20260806 05:02:29.986631 6360 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260806 05:02:29.990156 6378 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 05:02:29.991466 6360 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 05:02:29.991643 6360 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785992227050689-9601-0/minicluster-data/ts-0/data,/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785992227050689-9601-0/minicluster-data/ts-0/wal
uuid: "87fbe18fcd9b43de909b2ac042447d30"
format_stamp: "Formatted at 2026-08-06 05:02:29 on dist-test-slave-x4qh"
server_key: "3679d4564586d32fcd913222ebc1e68c"
server_key_iv: "7df72004d09007a404f4a71ca05175d2"
server_key_version: "encryptionkey@0"
I20260806 05:02:29.991875 6360 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785992227050689-9601-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785992227050689-9601-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785992227050689-9601-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 05:02:30.006160 6360 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 05:02:30.010452 6360 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 05:02:30.010795 6360 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 05:02:30.011833 6360 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 05:02:30.013142 6360 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 05:02:30.013248 6360 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 05:02:30.013391 6360 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 05:02:30.013474 6360 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 05:02:30.047037 6360 rpc_server.cc:307] RPC server started. Bound to: 127.9.96.65:42559
I20260806 05:02:30.047081 6539 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.96.65:42559 every 8 connection(s)
I20260806 05:02:30.048372 6360 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785992227050689-9601-0/minicluster-data/ts-0/data/info.pb
I20260806 05:02:30.057252 9601 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu as pid 6360
I20260806 05:02:30.057468 9601 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785992227050689-9601-0/minicluster-data/ts-0/wal/instance
I20260806 05:02:30.057937 9601 external_mini_cluster.cc:1428] Setting key 1c53fe7c6facf905e7bb1808c1ebcca6
Aug 06 05:02:30 dist-test-slave-x4qh krb5kdc[6200](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992549, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.96.65@KRBTEST.COM for kudu/127.9.96.126@KRBTEST.COM
I20260806 05:02:30.070873 6544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:02:30.051028 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.96.65:57779 (local address 127.9.96.126:38143)
0806 05:02:30.051575 (+ 547us) server_negotiation.cc:206] Beginning negotiation
0806 05:02:30.051580 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 05:02:30.052413 (+ 833us) server_negotiation.cc:407] Connection header received
0806 05:02:30.054387 (+ 1974us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 05:02:30.054394 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 05:02:30.054492 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 05:02:30.054621 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0806 05:02:30.056257 (+ 1636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:02:30.057247 (+ 990us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:02:30.059057 (+ 1810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:02:30.059585 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:02:30.059760 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:02:30.063286 (+ 3526us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 05:02:30.063296 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 05:02:30.063305 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 05:02:30.063353 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0806 05:02:30.067239 (+ 3886us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 05:02:30.068046 (+ 807us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 05:02:30.068055 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 05:02:30.068061 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 05:02:30.068241 (+ 180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 05:02:30.068708 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 05:02:30.068716 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 05:02:30.068723 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 05:02:30.069187 (+ 464us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 05:02:30.069371 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0806 05:02:30.070048 (+ 677us) server_negotiation.cc:299] Negotiation successful
0806 05:02:30.070319 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":433,"thread_start_us":298,"threads_started":1}
I20260806 05:02:30.071417 6545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:02:30.051470 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:38143 (local address 127.9.96.65:57779)
0806 05:02:30.052232 (+ 762us) negotiation.cc:107] Waiting for socket to connect
0806 05:02:30.052282 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0806 05:02:30.054084 (+ 1802us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 05:02:30.054860 (+ 776us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 05:02:30.054868 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 05:02:30.055455 (+ 587us) client_negotiation.cc:189] Negotiated authn=SASL
0806 05:02:30.056066 (+ 611us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 05:02:30.056085 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 05:02:30.057455 (+ 1370us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 05:02:30.057461 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 05:02:30.058863 (+ 1402us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 05:02:30.058877 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 05:02:30.059881 (+ 1004us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 05:02:30.059889 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 05:02:30.060158 (+ 269us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:02:30.060854 (+ 696us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 05:02:30.060876 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0806 05:02:30.063048 (+ 2172us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 05:02:30.067468 (+ 4420us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 05:02:30.067475 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 05:02:30.067487 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0806 05:02:30.067919 (+ 432us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 05:02:30.068376 (+ 457us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 05:02:30.068379 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 05:02:30.068382 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 05:02:30.068564 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 05:02:30.069346 (+ 782us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 05:02:30.069352 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 05:02:30.069638 (+ 286us) client_negotiation.cc:769] Sending connection context
0806 05:02:30.069861 (+ 223us) client_negotiation.cc:240] Negotiation successful
0806 05:02:30.070255 (+ 394us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":527,"thread_start_us":304,"threads_started":1}
I20260806 05:02:30.073096 6541 heartbeater.cc:347] Connected to a master server at 127.9.96.126:38143
I20260806 05:02:30.073438 6541 heartbeater.cc:464] Registering TS with master...
I20260806 05:02:30.074208 6541 heartbeater.cc:511] Master 127.9.96.126:38143 requested a full tablet report, sending...
I20260806 05:02:30.076557 6262 ts_manager.cc:194] Registered new tserver with Master: 87fbe18fcd9b43de909b2ac042447d30 (127.9.96.65:42559)
I20260806 05:02:30.078832 6262 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.96.65@KRBTEST.COM'} at 127.9.96.65:57779
WARNING: no policy specified for kudu/127.9.96.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.96.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.96.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.66.keytab.
Entry for principal kudu/127.9.96.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.66.keytab.
I20260806 05:02:30.127188 9601 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
/tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785992227050689-9601-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785992227050689-9601-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785992227050689-9601-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785992227050689-9601-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.96.66:0
--local_ip_for_outbound_sockets=127.9.96.66
--webserver_interface=127.9.96.66
--webserver_port=0
--tserver_master_addrs=127.9.96.126:38143
--keytab_file=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.66.keytab
--principal=kudu/127.9.96.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.96.84:44643
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785992227050689-9601-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785992227050689-9601-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785992227050689-9601-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 05:02:30.466825 6550 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 05:02:30.467082 6550 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 05:02:30.467227 6550 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 05:02:30.475211 6550 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 05:02:30.475309 6550 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 05:02:30.475533 6550 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.96.66
I20260806 05:02:30.485241 6550 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.96.84:44643
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785992227050689-9601-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785992227050689-9601-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.66.keytab
--principal=kudu/127.9.96.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.96.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785992227050689-9601-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.96.66
--webserver_port=0
--tserver_master_addrs=127.9.96.126:38143
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.96.66
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785992227050689-9601-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 03:56:12 UTC on ccycloud.quasar-guijbm.root.comops.site
build id 4569618
ASAN enabled
I20260806 05:02:30.486414 6550 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 05:02:30.487752 6550 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 05:02:30.501125 6559 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 05:02:30.501125 6560 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 05:02:30.501068 6567 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 05:02:30.502264 6550 server_base.cc:1034] running on GCE node
I20260806 05:02:30.502846 6550 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 05:02:30.503755 6550 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 05:02:30.504951 6550 hybrid_clock.cc:648] HybridClock initialized: now 1785992550504917 us; error 30 us; skew 500 ppm
I20260806 05:02:30.505215 6550 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 05:02:30 dist-test-slave-x4qh krb5kdc[6200](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992550, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.96.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 05:02:30.508484 6550 init.cc:377] Logged in from keytab as kudu/127.9.96.66@KRBTEST.COM (short username kudu)
I20260806 05:02:30.509948 6550 webserver.cc:460] Webserver started at http://127.9.96.66:36255/ using document root <none> and password file <none>
I20260806 05:02:30.510602 6550 fs_manager.cc:359] Metadata directory not provided
I20260806 05:02:30.510726 6550 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 05:02:30.511029 6550 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 05:02:30.513880 6550 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785992227050689-9601-0/minicluster-data/ts-1/data/instance:
uuid: "44df9ef59c454d859137bc6c13cc332f"
format_stamp: "Formatted at 2026-08-06 05:02:30 on dist-test-slave-x4qh"
server_key: "3b773b96e760fce5a219c32925846f88"
server_key_iv: "c9066b35a30093b44abbb68ccdc8067c"
server_key_version: "encryptionkey@0"
I20260806 05:02:30.514658 6550 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785992227050689-9601-0/minicluster-data/ts-1/wal/instance:
uuid: "44df9ef59c454d859137bc6c13cc332f"
format_stamp: "Formatted at 2026-08-06 05:02:30 on dist-test-slave-x4qh"
server_key: "3b773b96e760fce5a219c32925846f88"
server_key_iv: "c9066b35a30093b44abbb68ccdc8067c"
server_key_version: "encryptionkey@0"
I20260806 05:02:30.519567 6550 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260806 05:02:30.522765 6577 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 05:02:30.524092 6550 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 05:02:30.524451 6550 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785992227050689-9601-0/minicluster-data/ts-1/data,/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785992227050689-9601-0/minicluster-data/ts-1/wal
uuid: "44df9ef59c454d859137bc6c13cc332f"
format_stamp: "Formatted at 2026-08-06 05:02:30 on dist-test-slave-x4qh"
server_key: "3b773b96e760fce5a219c32925846f88"
server_key_iv: "c9066b35a30093b44abbb68ccdc8067c"
server_key_version: "encryptionkey@0"
I20260806 05:02:30.524730 6550 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785992227050689-9601-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785992227050689-9601-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785992227050689-9601-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 05:02:30.543412 6550 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 05:02:30.547823 6550 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 05:02:30.548146 6550 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 05:02:30.549126 6550 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 05:02:30.550555 6550 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 05:02:30.550684 6550 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 05:02:30.550837 6550 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 05:02:30.550916 6550 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 05:02:30.587116 6550 rpc_server.cc:307] RPC server started. Bound to: 127.9.96.66:42929
I20260806 05:02:30.587154 6737 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.96.66:42929 every 8 connection(s)
I20260806 05:02:30.588505 6550 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785992227050689-9601-0/minicluster-data/ts-1/data/info.pb
I20260806 05:02:30.595145 9601 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu as pid 6550
I20260806 05:02:30.595404 9601 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785992227050689-9601-0/minicluster-data/ts-1/wal/instance
I20260806 05:02:30.595992 9601 external_mini_cluster.cc:1428] Setting key 115d11bccd4ad6cf8833e9030fae45a2
Aug 06 05:02:30 dist-test-slave-x4qh krb5kdc[6200](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992550, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.96.66@KRBTEST.COM for kudu/127.9.96.126@KRBTEST.COM
I20260806 05:02:30.610917 6743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:02:30.591117 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.96.66:44797 (local address 127.9.96.126:38143)
0806 05:02:30.591618 (+ 501us) server_negotiation.cc:206] Beginning negotiation
0806 05:02:30.591625 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 05:02:30.592870 (+ 1245us) server_negotiation.cc:407] Connection header received
0806 05:02:30.594652 (+ 1782us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 05:02:30.594656 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 05:02:30.594735 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 05:02:30.594835 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0806 05:02:30.596929 (+ 2094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:02:30.598006 (+ 1077us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:02:30.600491 (+ 2485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:02:30.600962 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:02:30.601076 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:02:30.605319 (+ 4243us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 05:02:30.605323 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 05:02:30.605327 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 05:02:30.605358 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0806 05:02:30.607913 (+ 2555us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 05:02:30.608635 (+ 722us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 05:02:30.608639 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 05:02:30.608647 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0806 05:02:30.608799 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 05:02:30.609252 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 05:02:30.609256 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 05:02:30.609258 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 05:02:30.609515 (+ 257us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 05:02:30.609665 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0806 05:02:30.610328 (+ 663us) server_negotiation.cc:299] Negotiation successful
0806 05:02:30.610557 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":365,"thread_start_us":248,"threads_started":1}
I20260806 05:02:30.611868 6746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:02:30.591824 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:38143 (local address 127.9.96.66:44797)
0806 05:02:30.592715 (+ 891us) negotiation.cc:107] Waiting for socket to connect
0806 05:02:30.592753 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0806 05:02:30.594393 (+ 1640us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 05:02:30.595126 (+ 733us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 05:02:30.595140 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 05:02:30.595944 (+ 804us) client_negotiation.cc:189] Negotiated authn=SASL
0806 05:02:30.596755 (+ 811us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 05:02:30.596775 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 05:02:30.598176 (+ 1401us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 05:02:30.598182 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 05:02:30.600329 (+ 2147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 05:02:30.600347 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 05:02:30.601210 (+ 863us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 05:02:30.601220 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 05:02:30.601544 (+ 324us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:02:30.602370 (+ 826us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 05:02:30.602410 (+ 40us) client_negotiation.cc:656] calling sasl_client_start()
0806 05:02:30.605075 (+ 2665us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 05:02:30.608043 (+ 2968us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 05:02:30.608051 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 05:02:30.608065 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0806 05:02:30.608481 (+ 416us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 05:02:30.608914 (+ 433us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 05:02:30.608919 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 05:02:30.608922 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 05:02:30.609124 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 05:02:30.609639 (+ 515us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 05:02:30.609646 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 05:02:30.609949 (+ 303us) client_negotiation.cc:769] Sending connection context
0806 05:02:30.610216 (+ 267us) client_negotiation.cc:240] Negotiation successful
0806 05:02:30.610699 (+ 483us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":567,"thread_start_us":378,"threads_started":1}
I20260806 05:02:30.613258 6738 heartbeater.cc:347] Connected to a master server at 127.9.96.126:38143
I20260806 05:02:30.613547 6738 heartbeater.cc:464] Registering TS with master...
I20260806 05:02:30.614302 6738 heartbeater.cc:511] Master 127.9.96.126:38143 requested a full tablet report, sending...
I20260806 05:02:30.615708 6262 ts_manager.cc:194] Registered new tserver with Master: 44df9ef59c454d859137bc6c13cc332f (127.9.96.66:42929)
I20260806 05:02:30.616523 6262 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.96.66@KRBTEST.COM'} at 127.9.96.66:44797
WARNING: no policy specified for kudu/127.9.96.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.96.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.96.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.67.keytab.
Entry for principal kudu/127.9.96.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.67.keytab.
I20260806 05:02:30.662424 9601 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
/tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785992227050689-9601-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785992227050689-9601-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785992227050689-9601-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785992227050689-9601-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.96.67:0
--local_ip_for_outbound_sockets=127.9.96.67
--webserver_interface=127.9.96.67
--webserver_port=0
--tserver_master_addrs=127.9.96.126:38143
--keytab_file=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.67.keytab
--principal=kudu/127.9.96.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.96.84:44643
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785992227050689-9601-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785992227050689-9601-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785992227050689-9601-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 05:02:30.996295 6751 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 05:02:30.996583 6751 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 05:02:30.996778 6751 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 05:02:31.003962 6751 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 05:02:31.004065 6751 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 05:02:31.004236 6751 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.96.67
I20260806 05:02:31.013912 6751 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.96.84:44643
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785992227050689-9601-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785992227050689-9601-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.67.keytab
--principal=kudu/127.9.96.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.96.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785992227050689-9601-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.96.67
--webserver_port=0
--tserver_master_addrs=127.9.96.126:38143
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.96.67
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785992227050689-9601-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 03:56:12 UTC on ccycloud.quasar-guijbm.root.comops.site
build id 4569618
ASAN enabled
I20260806 05:02:31.015163 6751 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 05:02:31.016455 6751 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 05:02:31.030295 6762 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 05:02:31.030316 6760 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 05:02:31.030365 6759 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 05:02:31.031025 6751 server_base.cc:1034] running on GCE node
I20260806 05:02:31.031661 6751 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 05:02:31.032604 6751 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 05:02:31.033828 6751 hybrid_clock.cc:648] HybridClock initialized: now 1785992551033781 us; error 42 us; skew 500 ppm
I20260806 05:02:31.034135 6751 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 05:02:31 dist-test-slave-x4qh krb5kdc[6200](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992551, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.96.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 05:02:31.037286 6751 init.cc:377] Logged in from keytab as kudu/127.9.96.67@KRBTEST.COM (short username kudu)
I20260806 05:02:31.038625 6751 webserver.cc:460] Webserver started at http://127.9.96.67:41831/ using document root <none> and password file <none>
I20260806 05:02:31.039245 6751 fs_manager.cc:359] Metadata directory not provided
I20260806 05:02:31.039345 6751 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 05:02:31.039664 6751 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 05:02:31.042412 6751 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785992227050689-9601-0/minicluster-data/ts-2/data/instance:
uuid: "668f39d5e82a4671a8bbfa2d8f23b862"
format_stamp: "Formatted at 2026-08-06 05:02:31 on dist-test-slave-x4qh"
server_key: "5d9e034517726283f3e1a09d2b5ec382"
server_key_iv: "e1d1413c14e983b234a0a5739903c30f"
server_key_version: "encryptionkey@0"
I20260806 05:02:31.043143 6751 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785992227050689-9601-0/minicluster-data/ts-2/wal/instance:
uuid: "668f39d5e82a4671a8bbfa2d8f23b862"
format_stamp: "Formatted at 2026-08-06 05:02:31 on dist-test-slave-x4qh"
server_key: "5d9e034517726283f3e1a09d2b5ec382"
server_key_iv: "e1d1413c14e983b234a0a5739903c30f"
server_key_version: "encryptionkey@0"
I20260806 05:02:31.048312 6751 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.001s
I20260806 05:02:31.051815 6770 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 05:02:31.053105 6751 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260806 05:02:31.053285 6751 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785992227050689-9601-0/minicluster-data/ts-2/data,/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785992227050689-9601-0/minicluster-data/ts-2/wal
uuid: "668f39d5e82a4671a8bbfa2d8f23b862"
format_stamp: "Formatted at 2026-08-06 05:02:31 on dist-test-slave-x4qh"
server_key: "5d9e034517726283f3e1a09d2b5ec382"
server_key_iv: "e1d1413c14e983b234a0a5739903c30f"
server_key_version: "encryptionkey@0"
I20260806 05:02:31.053500 6751 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785992227050689-9601-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785992227050689-9601-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785992227050689-9601-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 05:02:31.070776 6751 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 05:02:31.074990 6751 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 05:02:31.075296 6751 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 05:02:31.076242 6751 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 05:02:31.077600 6751 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 05:02:31.077703 6751 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 05:02:31.077862 6751 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 05:02:31.077931 6751 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 05:02:31.082036 6541 heartbeater.cc:503] Master 127.9.96.126:38143 was elected leader, sending a full tablet report...
I20260806 05:02:31.112186 6751 rpc_server.cc:307] RPC server started. Bound to: 127.9.96.67:41151
I20260806 05:02:31.112296 6941 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.96.67:41151 every 8 connection(s)
I20260806 05:02:31.113489 6751 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785992227050689-9601-0/minicluster-data/ts-2/data/info.pb
I20260806 05:02:31.120450 9601 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu as pid 6751
I20260806 05:02:31.120639 9601 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785992227050689-9601-0/minicluster-data/ts-2/wal/instance
I20260806 05:02:31.121078 9601 external_mini_cluster.cc:1428] Setting key 77b4296f3d5848a9d9cb8ab70174e9a8
Aug 06 05:02:31 dist-test-slave-x4qh krb5kdc[6200](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992551, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.96.67@KRBTEST.COM for kudu/127.9.96.126@KRBTEST.COM
I20260806 05:02:31.133695 6944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:02:31.116258 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.96.67:52767 (local address 127.9.96.126:38143)
0806 05:02:31.116717 (+ 459us) server_negotiation.cc:206] Beginning negotiation
0806 05:02:31.116721 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 05:02:31.117729 (+ 1008us) server_negotiation.cc:407] Connection header received
0806 05:02:31.119557 (+ 1828us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 05:02:31.119562 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 05:02:31.119649 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 05:02:31.119751 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0806 05:02:31.121396 (+ 1645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:02:31.122231 (+ 835us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:02:31.123955 (+ 1724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:02:31.124341 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:02:31.124502 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:02:31.127924 (+ 3422us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 05:02:31.127933 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 05:02:31.127939 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 05:02:31.127981 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0806 05:02:31.130689 (+ 2708us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 05:02:31.131420 (+ 731us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 05:02:31.131426 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 05:02:31.131430 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 05:02:31.131561 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 05:02:31.131984 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 05:02:31.131989 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 05:02:31.131993 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 05:02:31.132270 (+ 277us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 05:02:31.132440 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0806 05:02:31.133073 (+ 633us) server_negotiation.cc:299] Negotiation successful
0806 05:02:31.133293 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":352,"thread_start_us":236,"threads_started":1}
I20260806 05:02:31.134684 6945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:02:31.116878 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:38143 (local address 127.9.96.67:52767)
0806 05:02:31.117576 (+ 698us) negotiation.cc:107] Waiting for socket to connect
0806 05:02:31.117613 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0806 05:02:31.119252 (+ 1639us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 05:02:31.119991 (+ 739us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 05:02:31.119998 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 05:02:31.120566 (+ 568us) client_negotiation.cc:189] Negotiated authn=SASL
0806 05:02:31.121224 (+ 658us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 05:02:31.121249 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 05:02:31.122392 (+ 1143us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 05:02:31.122397 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 05:02:31.123821 (+ 1424us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 05:02:31.123833 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 05:02:31.124613 (+ 780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 05:02:31.124621 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 05:02:31.124862 (+ 241us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:02:31.125522 (+ 660us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 05:02:31.125549 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0806 05:02:31.127722 (+ 2173us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 05:02:31.130830 (+ 3108us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 05:02:31.130839 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 05:02:31.130854 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0806 05:02:31.131257 (+ 403us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 05:02:31.131671 (+ 414us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 05:02:31.131676 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 05:02:31.131679 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 05:02:31.131864 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 05:02:31.132406 (+ 542us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 05:02:31.132413 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 05:02:31.132725 (+ 312us) client_negotiation.cc:769] Sending connection context
0806 05:02:31.132980 (+ 255us) client_negotiation.cc:240] Negotiation successful
0806 05:02:31.133489 (+ 509us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":473,"thread_start_us":280,"threads_started":1}
I20260806 05:02:31.136101 6942 heartbeater.cc:347] Connected to a master server at 127.9.96.126:38143
I20260806 05:02:31.136415 6942 heartbeater.cc:464] Registering TS with master...
I20260806 05:02:31.137107 6942 heartbeater.cc:511] Master 127.9.96.126:38143 requested a full tablet report, sending...
I20260806 05:02:31.138499 6262 ts_manager.cc:194] Registered new tserver with Master: 668f39d5e82a4671a8bbfa2d8f23b862 (127.9.96.67:41151)
I20260806 05:02:31.139297 6262 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.96.67@KRBTEST.COM'} at 127.9.96.67:52767
I20260806 05:02:31.148316 9601 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 05:02:31.165366 6944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:02:31.152638 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54570 (local address 127.9.96.126:38143)
0806 05:02:31.152873 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0806 05:02:31.152885 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 05:02:31.153311 (+ 426us) server_negotiation.cc:407] Connection header received
0806 05:02:31.153441 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 05:02:31.153445 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 05:02:31.153520 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 05:02:31.153605 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0806 05:02:31.154917 (+ 1312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:02:31.155862 (+ 945us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:02:31.157257 (+ 1395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:02:31.157767 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:02:31.157862 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:02:31.160296 (+ 2434us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 05:02:31.160300 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 05:02:31.160312 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 05:02:31.160341 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0806 05:02:31.162875 (+ 2534us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 05:02:31.163611 (+ 736us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 05:02:31.163615 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 05:02:31.163617 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 05:02:31.163745 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 05:02:31.164098 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 05:02:31.164102 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 05:02:31.164105 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 05:02:31.164479 (+ 374us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 05:02:31.164663 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0806 05:02:31.164980 (+ 317us) server_negotiation.cc:299] Negotiation successful
0806 05:02:31.165139 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":121}
I20260806 05:02:31.168948 6262 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54570:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 05:02:31.175918 6262 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 05:02:31.198861 6963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:02:31.188030 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55302 (local address 127.9.96.66:42929)
0806 05:02:31.190157 (+ 2127us) server_negotiation.cc:206] Beginning negotiation
0806 05:02:31.190163 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 05:02:31.190186 (+ 23us) server_negotiation.cc:407] Connection header received
0806 05:02:31.190275 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 05:02:31.190280 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 05:02:31.190557 (+ 277us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 05:02:31.191423 (+ 866us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 05:02:31.191601 (+ 178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:02:31.193275 (+ 1674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:02:31.196187 (+ 2912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:02:31.197654 (+ 1467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:02:31.197790 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:02:31.197973 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0806 05:02:31.198350 (+ 377us) server_negotiation.cc:299] Negotiation successful
0806 05:02:31.198507 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1994,"thread_start_us":1861,"threads_started":1}
I20260806 05:02:31.199156 6961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:02:31.187873 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.66:42929 (local address 127.0.0.1:55302)
0806 05:02:31.188782 (+ 909us) negotiation.cc:107] Waiting for socket to connect
0806 05:02:31.188839 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0806 05:02:31.189040 (+ 201us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 05:02:31.190821 (+ 1781us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 05:02:31.190828 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 05:02:31.190867 (+ 39us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 05:02:31.191189 (+ 322us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 05:02:31.191198 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 05:02:31.193421 (+ 2223us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 05:02:31.193425 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 05:02:31.195930 (+ 2505us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 05:02:31.195944 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 05:02:31.197892 (+ 1948us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 05:02:31.197897 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 05:02:31.198113 (+ 216us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:02:31.198150 (+ 37us) client_negotiation.cc:769] Sending connection context
0806 05:02:31.198294 (+ 144us) client_negotiation.cc:240] Negotiation successful
0806 05:02:31.198470 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":658,"thread_start_us":496,"threads_started":1}
I20260806 05:02:31.201076 6965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:02:31.189374 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.65:42559 (local address 127.0.0.1:34968)
0806 05:02:31.192671 (+ 3297us) negotiation.cc:107] Waiting for socket to connect
0806 05:02:31.192690 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0806 05:02:31.192883 (+ 193us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 05:02:31.193860 (+ 977us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 05:02:31.193873 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 05:02:31.193908 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 05:02:31.194422 (+ 514us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 05:02:31.194438 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 05:02:31.196341 (+ 1903us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 05:02:31.196349 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 05:02:31.197872 (+ 1523us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 05:02:31.197885 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 05:02:31.200089 (+ 2204us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 05:02:31.200095 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 05:02:31.200294 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:02:31.200406 (+ 112us) client_negotiation.cc:769] Sending connection context
0806 05:02:31.200503 (+ 97us) client_negotiation.cc:240] Negotiation successful
0806 05:02:31.200632 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3186,"mutex_wait_us":26,"thread_start_us":3030,"threads_started":1}
I20260806 05:02:31.201084 6966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:02:31.190101 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34968 (local address 127.9.96.65:42559)
0806 05:02:31.190854 (+ 753us) server_negotiation.cc:206] Beginning negotiation
0806 05:02:31.190864 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 05:02:31.192785 (+ 1921us) server_negotiation.cc:407] Connection header received
0806 05:02:31.193080 (+ 295us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 05:02:31.193088 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 05:02:31.193348 (+ 260us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 05:02:31.193524 (+ 176us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 05:02:31.194632 (+ 1108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:02:31.196169 (+ 1537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:02:31.198094 (+ 1925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:02:31.199577 (+ 1483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:02:31.199725 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:02:31.199947 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0806 05:02:31.200588 (+ 641us) server_negotiation.cc:299] Negotiation successful
0806 05:02:31.200754 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":569,"thread_start_us":397,"threads_started":1}
I20260806 05:02:31.201622 6964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:02:31.188599 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52324 (local address 127.9.96.67:41151)
0806 05:02:31.189713 (+ 1114us) server_negotiation.cc:206] Beginning negotiation
0806 05:02:31.189726 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 05:02:31.192386 (+ 2660us) server_negotiation.cc:407] Connection header received
0806 05:02:31.192692 (+ 306us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 05:02:31.192700 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 05:02:31.192902 (+ 202us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 05:02:31.193115 (+ 213us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 05:02:31.194130 (+ 1015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:02:31.195775 (+ 1645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:02:31.198409 (+ 2634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:02:31.199801 (+ 1392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:02:31.199896 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:02:31.200107 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0806 05:02:31.201168 (+ 1061us) server_negotiation.cc:299] Negotiation successful
0806 05:02:31.201340 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":908,"thread_start_us":768,"threads_started":1}
I20260806 05:02:31.202332 6962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:02:31.188028 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.67:41151 (local address 127.0.0.1:52324)
0806 05:02:31.192275 (+ 4247us) negotiation.cc:107] Waiting for socket to connect
0806 05:02:31.192296 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0806 05:02:31.192492 (+ 196us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 05:02:31.193418 (+ 926us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 05:02:31.193422 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 05:02:31.193448 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 05:02:31.193909 (+ 461us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 05:02:31.193921 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 05:02:31.195958 (+ 2037us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 05:02:31.195963 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 05:02:31.198231 (+ 2268us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 05:02:31.198246 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 05:02:31.200288 (+ 2042us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 05:02:31.200293 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 05:02:31.200914 (+ 621us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:02:31.200952 (+ 38us) client_negotiation.cc:769] Sending connection context
0806 05:02:31.201876 (+ 924us) client_negotiation.cc:240] Negotiation successful
0806 05:02:31.201965 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4113,"mutex_wait_us":386,"thread_start_us":3978,"threads_started":1}
I20260806 05:02:31.203325 6643 tablet_service.cc:1467] Processing CreateTablet for tablet 8272c7aa678449a68cf6e62814118e42 (DEFAULT_TABLE table=test-table [id=82b3bb6fa4954de3848271101dd66981]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 05:02:31.204672 6643 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8272c7aa678449a68cf6e62814118e42. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 05:02:31.204732 6448 tablet_service.cc:1467] Processing CreateTablet for tablet 8272c7aa678449a68cf6e62814118e42 (DEFAULT_TABLE table=test-table [id=82b3bb6fa4954de3848271101dd66981]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 05:02:31.206099 6448 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8272c7aa678449a68cf6e62814118e42. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 05:02:31.205892 6842 tablet_service.cc:1467] Processing CreateTablet for tablet 8272c7aa678449a68cf6e62814118e42 (DEFAULT_TABLE table=test-table [id=82b3bb6fa4954de3848271101dd66981]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 05:02:31.207258 6842 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8272c7aa678449a68cf6e62814118e42. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 05:02:31.215660 6967 tablet_bootstrap.cc:492] T 8272c7aa678449a68cf6e62814118e42 P 44df9ef59c454d859137bc6c13cc332f: Bootstrap starting.
I20260806 05:02:31.219944 6967 tablet_bootstrap.cc:654] T 8272c7aa678449a68cf6e62814118e42 P 44df9ef59c454d859137bc6c13cc332f: Neither blocks nor log segments found. Creating new log.
I20260806 05:02:31.220469 6975 tablet_bootstrap.cc:492] T 8272c7aa678449a68cf6e62814118e42 P 87fbe18fcd9b43de909b2ac042447d30: Bootstrap starting.
I20260806 05:02:31.221284 6967 log.cc:824] T 8272c7aa678449a68cf6e62814118e42 P 44df9ef59c454d859137bc6c13cc332f: Log is configured to *not* fsync() on all Append() calls
I20260806 05:02:31.221889 6976 tablet_bootstrap.cc:492] T 8272c7aa678449a68cf6e62814118e42 P 668f39d5e82a4671a8bbfa2d8f23b862: Bootstrap starting.
I20260806 05:02:31.224983 6976 tablet_bootstrap.cc:654] T 8272c7aa678449a68cf6e62814118e42 P 668f39d5e82a4671a8bbfa2d8f23b862: Neither blocks nor log segments found. Creating new log.
I20260806 05:02:31.225276 6975 tablet_bootstrap.cc:654] T 8272c7aa678449a68cf6e62814118e42 P 87fbe18fcd9b43de909b2ac042447d30: Neither blocks nor log segments found. Creating new log.
I20260806 05:02:31.225993 6976 log.cc:824] T 8272c7aa678449a68cf6e62814118e42 P 668f39d5e82a4671a8bbfa2d8f23b862: Log is configured to *not* fsync() on all Append() calls
I20260806 05:02:31.226418 6967 tablet_bootstrap.cc:492] T 8272c7aa678449a68cf6e62814118e42 P 44df9ef59c454d859137bc6c13cc332f: No bootstrap required, opened a new log
I20260806 05:02:31.226505 6975 log.cc:824] T 8272c7aa678449a68cf6e62814118e42 P 87fbe18fcd9b43de909b2ac042447d30: Log is configured to *not* fsync() on all Append() calls
I20260806 05:02:31.226749 6967 ts_tablet_manager.cc:1397] T 8272c7aa678449a68cf6e62814118e42 P 44df9ef59c454d859137bc6c13cc332f: Time spent bootstrapping tablet: real 0.011s user 0.006s sys 0.003s
I20260806 05:02:31.228503 6976 tablet_bootstrap.cc:492] T 8272c7aa678449a68cf6e62814118e42 P 668f39d5e82a4671a8bbfa2d8f23b862: No bootstrap required, opened a new log
I20260806 05:02:31.228782 6976 ts_tablet_manager.cc:1397] T 8272c7aa678449a68cf6e62814118e42 P 668f39d5e82a4671a8bbfa2d8f23b862: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260806 05:02:31.234392 6967 raft_consensus.cc:348] T 8272c7aa678449a68cf6e62814118e42 P 44df9ef59c454d859137bc6c13cc332f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44df9ef59c454d859137bc6c13cc332f" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 42929 } } peers { permanent_uuid: "668f39d5e82a4671a8bbfa2d8f23b862" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 41151 } } peers { permanent_uuid: "87fbe18fcd9b43de909b2ac042447d30" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 42559 } }
I20260806 05:02:31.234747 6967 raft_consensus.cc:374] T 8272c7aa678449a68cf6e62814118e42 P 44df9ef59c454d859137bc6c13cc332f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 05:02:31.234911 6967 raft_consensus.cc:729] T 8272c7aa678449a68cf6e62814118e42 P 44df9ef59c454d859137bc6c13cc332f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 44df9ef59c454d859137bc6c13cc332f, State: Initialized, Role: FOLLOWER
I20260806 05:02:31.235597 6967 consensus_queue.cc:260] T 8272c7aa678449a68cf6e62814118e42 P 44df9ef59c454d859137bc6c13cc332f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44df9ef59c454d859137bc6c13cc332f" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 42929 } } peers { permanent_uuid: "668f39d5e82a4671a8bbfa2d8f23b862" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 41151 } } peers { permanent_uuid: "87fbe18fcd9b43de909b2ac042447d30" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 42559 } }
I20260806 05:02:31.236302 6976 raft_consensus.cc:348] T 8272c7aa678449a68cf6e62814118e42 P 668f39d5e82a4671a8bbfa2d8f23b862 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44df9ef59c454d859137bc6c13cc332f" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 42929 } } peers { permanent_uuid: "668f39d5e82a4671a8bbfa2d8f23b862" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 41151 } } peers { permanent_uuid: "87fbe18fcd9b43de909b2ac042447d30" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 42559 } }
I20260806 05:02:31.236739 6976 raft_consensus.cc:374] T 8272c7aa678449a68cf6e62814118e42 P 668f39d5e82a4671a8bbfa2d8f23b862 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 05:02:31.236747 6738 heartbeater.cc:503] Master 127.9.96.126:38143 was elected leader, sending a full tablet report...
I20260806 05:02:31.236857 6976 raft_consensus.cc:729] T 8272c7aa678449a68cf6e62814118e42 P 668f39d5e82a4671a8bbfa2d8f23b862 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 668f39d5e82a4671a8bbfa2d8f23b862, State: Initialized, Role: FOLLOWER
I20260806 05:02:31.237353 6975 tablet_bootstrap.cc:492] T 8272c7aa678449a68cf6e62814118e42 P 87fbe18fcd9b43de909b2ac042447d30: No bootstrap required, opened a new log
I20260806 05:02:31.237433 6976 consensus_queue.cc:260] T 8272c7aa678449a68cf6e62814118e42 P 668f39d5e82a4671a8bbfa2d8f23b862 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44df9ef59c454d859137bc6c13cc332f" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 42929 } } peers { permanent_uuid: "668f39d5e82a4671a8bbfa2d8f23b862" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 41151 } } peers { permanent_uuid: "87fbe18fcd9b43de909b2ac042447d30" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 42559 } }
I20260806 05:02:31.237648 6975 ts_tablet_manager.cc:1397] T 8272c7aa678449a68cf6e62814118e42 P 87fbe18fcd9b43de909b2ac042447d30: Time spent bootstrapping tablet: real 0.017s user 0.000s sys 0.006s
I20260806 05:02:31.238469 6942 heartbeater.cc:503] Master 127.9.96.126:38143 was elected leader, sending a full tablet report...
I20260806 05:02:31.238818 6967 ts_tablet_manager.cc:1428] T 8272c7aa678449a68cf6e62814118e42 P 44df9ef59c454d859137bc6c13cc332f: Time spent starting tablet: real 0.012s user 0.007s sys 0.003s
I20260806 05:02:31.239110 6976 ts_tablet_manager.cc:1428] T 8272c7aa678449a68cf6e62814118e42 P 668f39d5e82a4671a8bbfa2d8f23b862: Time spent starting tablet: real 0.010s user 0.011s sys 0.000s
I20260806 05:02:31.247850 6975 raft_consensus.cc:348] T 8272c7aa678449a68cf6e62814118e42 P 87fbe18fcd9b43de909b2ac042447d30 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44df9ef59c454d859137bc6c13cc332f" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 42929 } } peers { permanent_uuid: "668f39d5e82a4671a8bbfa2d8f23b862" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 41151 } } peers { permanent_uuid: "87fbe18fcd9b43de909b2ac042447d30" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 42559 } }
I20260806 05:02:31.248157 6975 raft_consensus.cc:374] T 8272c7aa678449a68cf6e62814118e42 P 87fbe18fcd9b43de909b2ac042447d30 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 05:02:31.248224 6975 raft_consensus.cc:729] T 8272c7aa678449a68cf6e62814118e42 P 87fbe18fcd9b43de909b2ac042447d30 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 87fbe18fcd9b43de909b2ac042447d30, State: Initialized, Role: FOLLOWER
I20260806 05:02:31.248663 6975 consensus_queue.cc:260] T 8272c7aa678449a68cf6e62814118e42 P 87fbe18fcd9b43de909b2ac042447d30 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44df9ef59c454d859137bc6c13cc332f" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 42929 } } peers { permanent_uuid: "668f39d5e82a4671a8bbfa2d8f23b862" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 41151 } } peers { permanent_uuid: "87fbe18fcd9b43de909b2ac042447d30" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 42559 } }
I20260806 05:02:31.250023 6975 ts_tablet_manager.cc:1428] T 8272c7aa678449a68cf6e62814118e42 P 87fbe18fcd9b43de909b2ac042447d30: Time spent starting tablet: real 0.012s user 0.008s sys 0.003s
W20260806 05:02:31.300676 6543 tablet.cc:2369] T 8272c7aa678449a68cf6e62814118e42 P 87fbe18fcd9b43de909b2ac042447d30: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 05:02:31.330785 6983 raft_consensus.cc:482] T 8272c7aa678449a68cf6e62814118e42 P 44df9ef59c454d859137bc6c13cc332f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 05:02:31.331066 6983 raft_consensus.cc:504] T 8272c7aa678449a68cf6e62814118e42 P 44df9ef59c454d859137bc6c13cc332f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44df9ef59c454d859137bc6c13cc332f" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 42929 } } peers { permanent_uuid: "668f39d5e82a4671a8bbfa2d8f23b862" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 41151 } } peers { permanent_uuid: "87fbe18fcd9b43de909b2ac042447d30" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 42559 } }
I20260806 05:02:31.332784 6983 leader_election.cc:288] T 8272c7aa678449a68cf6e62814118e42 P 44df9ef59c454d859137bc6c13cc332f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 668f39d5e82a4671a8bbfa2d8f23b862 (127.9.96.67:41151), 87fbe18fcd9b43de909b2ac042447d30 (127.9.96.65:42559)
I20260806 05:02:31.339834 6966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:02:31.333251 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.96.66:51581 (local address 127.9.96.65:42559)
0806 05:02:31.333464 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0806 05:02:31.333470 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 05:02:31.334507 (+ 1037us) server_negotiation.cc:407] Connection header received
0806 05:02:31.334720 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 05:02:31.334726 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 05:02:31.334794 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 05:02:31.334892 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 05:02:31.335503 (+ 611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:02:31.336217 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:02:31.338135 (+ 1918us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:02:31.338901 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:02:31.339055 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:02:31.339163 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0806 05:02:31.339440 (+ 277us) server_negotiation.cc:299] Negotiation successful
0806 05:02:31.339573 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260806 05:02:31.339999 6988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:02:31.333247 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.65:42559 (local address 127.9.96.66:51581)
0806 05:02:31.334384 (+ 1137us) negotiation.cc:107] Waiting for socket to connect
0806 05:02:31.334400 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 05:02:31.334569 (+ 169us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 05:02:31.334909 (+ 340us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 05:02:31.334913 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 05:02:31.334937 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 05:02:31.335325 (+ 388us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 05:02:31.335335 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 05:02:31.336353 (+ 1018us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 05:02:31.336360 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 05:02:31.337977 (+ 1617us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 05:02:31.337993 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 05:02:31.339119 (+ 1126us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 05:02:31.339125 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 05:02:31.339270 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:02:31.339292 (+ 22us) client_negotiation.cc:769] Sending connection context
0806 05:02:31.339426 (+ 134us) client_negotiation.cc:240] Negotiation successful
0806 05:02:31.339568 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":967,"mutex_wait_us":57,"thread_start_us":576,"threads_started":1}
I20260806 05:02:31.340298 6964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:02:31.333357 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.96.66:32987 (local address 127.9.96.67:41151)
0806 05:02:31.333549 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0806 05:02:31.333571 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0806 05:02:31.334578 (+ 1007us) server_negotiation.cc:407] Connection header received
0806 05:02:31.334822 (+ 244us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 05:02:31.334826 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 05:02:31.334891 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 05:02:31.334998 (+ 107us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 05:02:31.335712 (+ 714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:02:31.336478 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:02:31.338432 (+ 1954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:02:31.339269 (+ 837us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:02:31.339442 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:02:31.339554 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0806 05:02:31.339872 (+ 318us) server_negotiation.cc:299] Negotiation successful
0806 05:02:31.340008 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260806 05:02:31.340399 6987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:02:31.333247 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.67:41151 (local address 127.9.96.66:32987)
0806 05:02:31.334447 (+ 1200us) negotiation.cc:107] Waiting for socket to connect
0806 05:02:31.334463 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 05:02:31.334642 (+ 179us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 05:02:31.335020 (+ 378us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 05:02:31.335024 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 05:02:31.335066 (+ 42us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 05:02:31.335548 (+ 482us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 05:02:31.335558 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 05:02:31.336677 (+ 1119us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 05:02:31.336681 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 05:02:31.338263 (+ 1582us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 05:02:31.338276 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 05:02:31.339511 (+ 1235us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 05:02:31.339517 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 05:02:31.339694 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:02:31.339719 (+ 25us) client_negotiation.cc:769] Sending connection context
0806 05:02:31.339843 (+ 124us) client_negotiation.cc:240] Negotiation successful
0806 05:02:31.339991 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1084,"mutex_wait_us":36,"thread_start_us":649,"threads_started":1}
W20260806 05:02:31.340622 6742 tablet.cc:2369] T 8272c7aa678449a68cf6e62814118e42 P 44df9ef59c454d859137bc6c13cc332f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 05:02:31.341508 6480 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8272c7aa678449a68cf6e62814118e42" candidate_uuid: "44df9ef59c454d859137bc6c13cc332f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "87fbe18fcd9b43de909b2ac042447d30" is_pre_election: true
I20260806 05:02:31.341672 6869 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8272c7aa678449a68cf6e62814118e42" candidate_uuid: "44df9ef59c454d859137bc6c13cc332f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "668f39d5e82a4671a8bbfa2d8f23b862" is_pre_election: true
I20260806 05:02:31.342099 6480 raft_consensus.cc:2454] T 8272c7aa678449a68cf6e62814118e42 P 87fbe18fcd9b43de909b2ac042447d30 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 44df9ef59c454d859137bc6c13cc332f in term 0.
I20260806 05:02:31.342217 6869 raft_consensus.cc:2454] T 8272c7aa678449a68cf6e62814118e42 P 668f39d5e82a4671a8bbfa2d8f23b862 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 44df9ef59c454d859137bc6c13cc332f in term 0.
I20260806 05:02:31.342926 6578 leader_election.cc:302] T 8272c7aa678449a68cf6e62814118e42 P 44df9ef59c454d859137bc6c13cc332f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 44df9ef59c454d859137bc6c13cc332f, 87fbe18fcd9b43de909b2ac042447d30; no voters:
I20260806 05:02:31.343284 6983 raft_consensus.cc:2781] T 8272c7aa678449a68cf6e62814118e42 P 44df9ef59c454d859137bc6c13cc332f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 05:02:31.343456 6983 raft_consensus.cc:482] T 8272c7aa678449a68cf6e62814118e42 P 44df9ef59c454d859137bc6c13cc332f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 05:02:31.343590 6983 raft_consensus.cc:3037] T 8272c7aa678449a68cf6e62814118e42 P 44df9ef59c454d859137bc6c13cc332f [term 0 FOLLOWER]: Advancing to term 1
I20260806 05:02:31.345803 6983 raft_consensus.cc:504] T 8272c7aa678449a68cf6e62814118e42 P 44df9ef59c454d859137bc6c13cc332f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44df9ef59c454d859137bc6c13cc332f" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 42929 } } peers { permanent_uuid: "668f39d5e82a4671a8bbfa2d8f23b862" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 41151 } } peers { permanent_uuid: "87fbe18fcd9b43de909b2ac042447d30" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 42559 } }
I20260806 05:02:31.346397 6983 leader_election.cc:288] T 8272c7aa678449a68cf6e62814118e42 P 44df9ef59c454d859137bc6c13cc332f [CANDIDATE]: Term 1 election: Requested vote from peers 668f39d5e82a4671a8bbfa2d8f23b862 (127.9.96.67:41151), 87fbe18fcd9b43de909b2ac042447d30 (127.9.96.65:42559)
I20260806 05:02:31.346848 6869 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8272c7aa678449a68cf6e62814118e42" candidate_uuid: "44df9ef59c454d859137bc6c13cc332f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "668f39d5e82a4671a8bbfa2d8f23b862"
I20260806 05:02:31.346871 6480 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8272c7aa678449a68cf6e62814118e42" candidate_uuid: "44df9ef59c454d859137bc6c13cc332f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "87fbe18fcd9b43de909b2ac042447d30"
I20260806 05:02:31.347026 6869 raft_consensus.cc:3037] T 8272c7aa678449a68cf6e62814118e42 P 668f39d5e82a4671a8bbfa2d8f23b862 [term 0 FOLLOWER]: Advancing to term 1
I20260806 05:02:31.347085 6480 raft_consensus.cc:3037] T 8272c7aa678449a68cf6e62814118e42 P 87fbe18fcd9b43de909b2ac042447d30 [term 0 FOLLOWER]: Advancing to term 1
I20260806 05:02:31.349408 6869 raft_consensus.cc:2454] T 8272c7aa678449a68cf6e62814118e42 P 668f39d5e82a4671a8bbfa2d8f23b862 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 44df9ef59c454d859137bc6c13cc332f in term 1.
I20260806 05:02:31.349457 6480 raft_consensus.cc:2454] T 8272c7aa678449a68cf6e62814118e42 P 87fbe18fcd9b43de909b2ac042447d30 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 44df9ef59c454d859137bc6c13cc332f in term 1.
I20260806 05:02:31.349843 6580 leader_election.cc:302] T 8272c7aa678449a68cf6e62814118e42 P 44df9ef59c454d859137bc6c13cc332f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 44df9ef59c454d859137bc6c13cc332f, 668f39d5e82a4671a8bbfa2d8f23b862; no voters:
I20260806 05:02:31.350185 6983 raft_consensus.cc:2781] T 8272c7aa678449a68cf6e62814118e42 P 44df9ef59c454d859137bc6c13cc332f [term 1 FOLLOWER]: Leader election won for term 1
I20260806 05:02:31.350651 6983 raft_consensus.cc:686] T 8272c7aa678449a68cf6e62814118e42 P 44df9ef59c454d859137bc6c13cc332f [term 1 LEADER]: Becoming Leader. State: Replica: 44df9ef59c454d859137bc6c13cc332f, State: Running, Role: LEADER
I20260806 05:02:31.351155 6983 consensus_queue.cc:237] T 8272c7aa678449a68cf6e62814118e42 P 44df9ef59c454d859137bc6c13cc332f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44df9ef59c454d859137bc6c13cc332f" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 42929 } } peers { permanent_uuid: "668f39d5e82a4671a8bbfa2d8f23b862" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 41151 } } peers { permanent_uuid: "87fbe18fcd9b43de909b2ac042447d30" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 42559 } }
I20260806 05:02:31.355830 6262 catalog_manager.cc:5495] T 8272c7aa678449a68cf6e62814118e42 P 44df9ef59c454d859137bc6c13cc332f reported cstate change: term changed from 0 to 1, leader changed from <none> to 44df9ef59c454d859137bc6c13cc332f (127.9.96.66). New cstate: current_term: 1 leader_uuid: "44df9ef59c454d859137bc6c13cc332f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44df9ef59c454d859137bc6c13cc332f" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 42929 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "668f39d5e82a4671a8bbfa2d8f23b862" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 41151 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "87fbe18fcd9b43de909b2ac042447d30" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 42559 } health_report { overall_health: UNKNOWN } } }
W20260806 05:02:31.365059 6943 tablet.cc:2369] T 8272c7aa678449a68cf6e62814118e42 P 668f39d5e82a4671a8bbfa2d8f23b862: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 05:02:31.403609 6963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:02:31.397954 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55316 (local address 127.9.96.66:42929)
0806 05:02:31.398146 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0806 05:02:31.398151 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 05:02:31.398265 (+ 114us) server_negotiation.cc:407] Connection header received
0806 05:02:31.398447 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 05:02:31.398451 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 05:02:31.398542 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 05:02:31.398672 (+ 130us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 05:02:31.399170 (+ 498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:02:31.399976 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:02:31.401499 (+ 1523us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:02:31.401890 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:02:31.401983 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:02:31.402475 (+ 492us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 05:02:31.402991 (+ 516us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 05:02:31.403099 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0806 05:02:31.403237 (+ 138us) server_negotiation.cc:299] Negotiation successful
0806 05:02:31.403306 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260806 05:02:31.411633 6480 raft_consensus.cc:1261] T 8272c7aa678449a68cf6e62814118e42 P 87fbe18fcd9b43de909b2ac042447d30 [term 1 FOLLOWER]: Refusing update from remote peer 44df9ef59c454d859137bc6c13cc332f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 05:02:31.411669 6869 raft_consensus.cc:1261] T 8272c7aa678449a68cf6e62814118e42 P 668f39d5e82a4671a8bbfa2d8f23b862 [term 1 FOLLOWER]: Refusing update from remote peer 44df9ef59c454d859137bc6c13cc332f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 05:02:31.412601 6989 consensus_queue.cc:1035] T 8272c7aa678449a68cf6e62814118e42 P 44df9ef59c454d859137bc6c13cc332f [LEADER]: Connected to new peer: Peer: permanent_uuid: "87fbe18fcd9b43de909b2ac042447d30" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 42559 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 05:02:31.412951 6983 consensus_queue.cc:1035] T 8272c7aa678449a68cf6e62814118e42 P 44df9ef59c454d859137bc6c13cc332f [LEADER]: Connected to new peer: Peer: permanent_uuid: "668f39d5e82a4671a8bbfa2d8f23b862" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 41151 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 05:02:31.438934 6262 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54570:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 05:02:31.439198 6262 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54570:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 05:02:31.443111 6262 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9e006065b18b49318c102b46e1ced923: Sending DeleteTablet for 3 replicas of tablet 8272c7aa678449a68cf6e62814118e42
I20260806 05:02:31.444315 6448 tablet_service.cc:1514] Processing DeleteTablet for tablet 8272c7aa678449a68cf6e62814118e42 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 05:02:31 UTC) from {username='kudu', principal='kudu/127.9.96.126@KRBTEST.COM'} at 127.0.0.1:34968
I20260806 05:02:31.444469 6643 tablet_service.cc:1514] Processing DeleteTablet for tablet 8272c7aa678449a68cf6e62814118e42 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 05:02:31 UTC) from {username='kudu', principal='kudu/127.9.96.126@KRBTEST.COM'} at 127.0.0.1:55302
I20260806 05:02:31.444638 6842 tablet_service.cc:1514] Processing DeleteTablet for tablet 8272c7aa678449a68cf6e62814118e42 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 05:02:31 UTC) from {username='kudu', principal='kudu/127.9.96.126@KRBTEST.COM'} at 127.0.0.1:52324
I20260806 05:02:31.445811 9601 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu pid 6360
I20260806 05:02:31.461678 7008 tablet_replica.cc:330] T 8272c7aa678449a68cf6e62814118e42 P 87fbe18fcd9b43de909b2ac042447d30: stopping tablet replica
I20260806 05:02:31.463883 7008 raft_consensus.cc:2229] T 8272c7aa678449a68cf6e62814118e42 P 87fbe18fcd9b43de909b2ac042447d30 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 05:02:31.464545 7008 raft_consensus.cc:2258] T 8272c7aa678449a68cf6e62814118e42 P 87fbe18fcd9b43de909b2ac042447d30 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 05:02:31.465180 7010 tablet_replica.cc:330] T 8272c7aa678449a68cf6e62814118e42 P 44df9ef59c454d859137bc6c13cc332f: stopping tablet replica
I20260806 05:02:31.467321 7010 raft_consensus.cc:2229] T 8272c7aa678449a68cf6e62814118e42 P 44df9ef59c454d859137bc6c13cc332f [term 1 LEADER]: Raft consensus shutting down.
I20260806 05:02:31.468050 7010 raft_consensus.cc:2258] T 8272c7aa678449a68cf6e62814118e42 P 44df9ef59c454d859137bc6c13cc332f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 05:02:31.468225 7008 ts_tablet_manager.cc:1905] T 8272c7aa678449a68cf6e62814118e42 P 87fbe18fcd9b43de909b2ac042447d30: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 05:02:31.469013 7015 tablet_replica.cc:330] T 8272c7aa678449a68cf6e62814118e42 P 668f39d5e82a4671a8bbfa2d8f23b862: stopping tablet replica
I20260806 05:02:31.469375 7015 raft_consensus.cc:2229] T 8272c7aa678449a68cf6e62814118e42 P 668f39d5e82a4671a8bbfa2d8f23b862 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 05:02:31.469796 7015 raft_consensus.cc:2258] T 8272c7aa678449a68cf6e62814118e42 P 668f39d5e82a4671a8bbfa2d8f23b862 [term 1 FOLLOWER]: Raft consensus is shut down!
Aug 06 05:02:31 dist-test-slave-x4qh krb5kdc[6200](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992548, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.96.65@KRBTEST.COM
I20260806 05:02:31.472024 7015 ts_tablet_manager.cc:1905] T 8272c7aa678449a68cf6e62814118e42 P 668f39d5e82a4671a8bbfa2d8f23b862: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 05:02:31.474876 7010 ts_tablet_manager.cc:1905] T 8272c7aa678449a68cf6e62814118e42 P 44df9ef59c454d859137bc6c13cc332f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 05:02:31.477809 7008 ts_tablet_manager.cc:1918] T 8272c7aa678449a68cf6e62814118e42 P 87fbe18fcd9b43de909b2ac042447d30: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 05:02:31.478019 7008 log.cc:1196] T 8272c7aa678449a68cf6e62814118e42 P 87fbe18fcd9b43de909b2ac042447d30: Deleting WAL directory at /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785992227050689-9601-0/minicluster-data/ts-0/wal/wals/8272c7aa678449a68cf6e62814118e42
I20260806 05:02:31.478592 7008 ts_tablet_manager.cc:1939] T 8272c7aa678449a68cf6e62814118e42 P 87fbe18fcd9b43de909b2ac042447d30: Deleting consensus metadata
I20260806 05:02:31.479328 7015 ts_tablet_manager.cc:1918] T 8272c7aa678449a68cf6e62814118e42 P 668f39d5e82a4671a8bbfa2d8f23b862: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 05:02:31.479547 7015 log.cc:1196] T 8272c7aa678449a68cf6e62814118e42 P 668f39d5e82a4671a8bbfa2d8f23b862: Deleting WAL directory at /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785992227050689-9601-0/minicluster-data/ts-2/wal/wals/8272c7aa678449a68cf6e62814118e42
I20260806 05:02:31.480149 7015 ts_tablet_manager.cc:1939] T 8272c7aa678449a68cf6e62814118e42 P 668f39d5e82a4671a8bbfa2d8f23b862: Deleting consensus metadata
I20260806 05:02:31.480576 6239 catalog_manager.cc:4841] TS 87fbe18fcd9b43de909b2ac042447d30 (127.9.96.65:42559): tablet 8272c7aa678449a68cf6e62814118e42 (table test-table [id=82b3bb6fa4954de3848271101dd66981]) successfully deleted
I20260806 05:02:31.481786 6966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:02:31.447520 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34980 (local address 127.9.96.65:42559)
0806 05:02:31.447743 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0806 05:02:31.447763 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0806 05:02:31.458110 (+ 10347us) server_negotiation.cc:407] Connection header received
0806 05:02:31.458607 (+ 497us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 05:02:31.458617 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 05:02:31.458772 (+ 155us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 05:02:31.458868 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0806 05:02:31.461955 (+ 3087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:02:31.463457 (+ 1502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:02:31.465370 (+ 1913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:02:31.466051 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:02:31.466168 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:02:31.473233 (+ 7065us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 05:02:31.473248 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 05:02:31.473265 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 05:02:31.473307 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0806 05:02:31.477901 (+ 4594us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 05:02:31.478859 (+ 958us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 05:02:31.478868 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 05:02:31.478871 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 05:02:31.479056 (+ 185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 05:02:31.479629 (+ 573us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 05:02:31.479636 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 05:02:31.479640 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 05:02:31.480143 (+ 503us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 05:02:31.480468 (+ 325us) server_negotiation.cc:1035] Waiting for connection context
0806 05:02:31.481122 (+ 654us) server_negotiation.cc:299] Negotiation successful
0806 05:02:31.481350 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260806 05:02:31.481998 6241 catalog_manager.cc:4841] TS 668f39d5e82a4671a8bbfa2d8f23b862 (127.9.96.67:41151): tablet 8272c7aa678449a68cf6e62814118e42 (table test-table [id=82b3bb6fa4954de3848271101dd66981]) successfully deleted
I20260806 05:02:31.483119 7010 ts_tablet_manager.cc:1918] T 8272c7aa678449a68cf6e62814118e42 P 44df9ef59c454d859137bc6c13cc332f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 05:02:31.483314 7010 log.cc:1196] T 8272c7aa678449a68cf6e62814118e42 P 44df9ef59c454d859137bc6c13cc332f: Deleting WAL directory at /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785992227050689-9601-0/minicluster-data/ts-1/wal/wals/8272c7aa678449a68cf6e62814118e42
I20260806 05:02:31.483345 6534 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 05:02:31.483902 7010 ts_tablet_manager.cc:1939] T 8272c7aa678449a68cf6e62814118e42 P 44df9ef59c454d859137bc6c13cc332f: Deleting consensus metadata
I20260806 05:02:31.486220 6239 catalog_manager.cc:4841] TS 44df9ef59c454d859137bc6c13cc332f (127.9.96.66:42929): tablet 8272c7aa678449a68cf6e62814118e42 (table test-table [id=82b3bb6fa4954de3848271101dd66981]) successfully deleted
I20260806 05:02:31.577347 9601 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu with pid 6360
I20260806 05:02:31.609659 9601 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu pid 6550
Aug 06 05:02:31 dist-test-slave-x4qh krb5kdc[6200](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992548, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.96.66@KRBTEST.COM
I20260806 05:02:31.627164 6963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:02:31.610415 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55320 (local address 127.9.96.66:42929)
0806 05:02:31.610608 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0806 05:02:31.610614 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 05:02:31.610635 (+ 21us) server_negotiation.cc:407] Connection header received
0806 05:02:31.610780 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 05:02:31.610786 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 05:02:31.610866 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 05:02:31.610981 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0806 05:02:31.612377 (+ 1396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:02:31.613614 (+ 1237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:02:31.614852 (+ 1238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:02:31.615555 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:02:31.615686 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:02:31.620170 (+ 4484us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 05:02:31.620183 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 05:02:31.620202 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 05:02:31.620266 (+ 64us) server_negotiation.cc:922] calling sasl_server_start()
0806 05:02:31.624396 (+ 4130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 05:02:31.625007 (+ 611us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 05:02:31.625017 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 05:02:31.625020 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 05:02:31.625200 (+ 180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 05:02:31.625584 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 05:02:31.625589 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 05:02:31.625593 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 05:02:31.626028 (+ 435us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 05:02:31.626381 (+ 353us) server_negotiation.cc:1035] Waiting for connection context
0806 05:02:31.626549 (+ 168us) server_negotiation.cc:299] Negotiation successful
0806 05:02:31.626775 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260806 05:02:31.627943 6729 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 05:02:31.744150 9601 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu with pid 6550
I20260806 05:02:31.770699 9601 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu pid 6751
Aug 06 05:02:31 dist-test-slave-x4qh krb5kdc[6200](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992548, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.96.67@KRBTEST.COM
I20260806 05:02:31.786479 6964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:02:31.771532 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52334 (local address 127.9.96.67:41151)
0806 05:02:31.771785 (+ 253us) server_negotiation.cc:206] Beginning negotiation
0806 05:02:31.771792 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 05:02:31.771845 (+ 53us) server_negotiation.cc:407] Connection header received
0806 05:02:31.772255 (+ 410us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 05:02:31.772260 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 05:02:31.772348 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 05:02:31.772440 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0806 05:02:31.773866 (+ 1426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:02:31.774609 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:02:31.776189 (+ 1580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:02:31.776582 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:02:31.776678 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:02:31.781074 (+ 4396us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 05:02:31.781088 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 05:02:31.781104 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 05:02:31.781135 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0806 05:02:31.784072 (+ 2937us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 05:02:31.784716 (+ 644us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 05:02:31.784723 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 05:02:31.784725 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 05:02:31.784862 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 05:02:31.785175 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 05:02:31.785178 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 05:02:31.785180 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 05:02:31.785514 (+ 334us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 05:02:31.785739 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0806 05:02:31.785940 (+ 201us) server_negotiation.cc:299] Negotiation successful
0806 05:02:31.786094 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260806 05:02:31.787199 6932 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 05:02:31.874053 9601 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu with pid 6751
I20260806 05:02:31.895529 9601 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu pid 6219
I20260806 05:02:31.896718 6308 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 05:02:32.005887 9601 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu with pid 6219
2026-08-06T05:02:32Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (4635 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-08-06T05:02:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T05:02:32Z Disabled control of system clock
I20260806 05:02:32.101567 9601 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
/tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785992227050689-9601-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785992227050689-9601-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785992227050689-9601-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785992227050689-9601-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.96.126:35245
--webserver_interface=127.9.96.126
--webserver_port=0
--builtin_ntp_servers=127.9.96.84:36671
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.96.126:35245
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260806 05:02:32.440404 7026 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 05:02:32.440681 7026 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 05:02:32.440771 7026 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 05:02:32.447990 7026 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 05:02:32.448086 7026 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 05:02:32.448124 7026 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 05:02:32.448173 7026 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 05:02:32.448217 7026 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 05:02:32.457069 7026 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.96.84:36671
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785992227050689-9601-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785992227050689-9601-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.96.126:35245
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.96.126:35245
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785992227050689-9601-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.96.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785992227050689-9601-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 03:56:12 UTC on ccycloud.quasar-guijbm.root.comops.site
build id 4569618
ASAN enabled
I20260806 05:02:32.458210 7026 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 05:02:32.459939 7026 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 05:02:32.472036 7031 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 05:02:32.472061 7032 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 05:02:32.472030 7035 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 05:02:32.472905 7026 server_base.cc:1034] running on GCE node
I20260806 05:02:32.473963 7026 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 05:02:32.475339 7026 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 05:02:32.476677 7026 hybrid_clock.cc:648] HybridClock initialized: now 1785992552476564 us; error 98 us; skew 500 ppm
I20260806 05:02:32.477139 7026 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 05:02:32.479883 7026 webserver.cc:460] Webserver started at http://127.9.96.126:46731/ using document root <none> and password file <none>
I20260806 05:02:32.480522 7026 fs_manager.cc:359] Metadata directory not provided
I20260806 05:02:32.480611 7026 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 05:02:32.480880 7026 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 05:02:32.483774 7026 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785992227050689-9601-0/minicluster-data/master-0/data/instance:
uuid: "76c1246e08154426aaf147c008a100e9"
format_stamp: "Formatted at 2026-08-06 05:02:32 on dist-test-slave-x4qh"
server_key: "3f0874df315ef1923699df5fb84e7697"
server_key_iv: "e127d88837259f1fabd258e75358fb88"
server_key_version: "encryptionkey@0"
I20260806 05:02:32.484541 7026 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785992227050689-9601-0/minicluster-data/master-0/wal/instance:
uuid: "76c1246e08154426aaf147c008a100e9"
format_stamp: "Formatted at 2026-08-06 05:02:32 on dist-test-slave-x4qh"
server_key: "3f0874df315ef1923699df5fb84e7697"
server_key_iv: "e127d88837259f1fabd258e75358fb88"
server_key_version: "encryptionkey@0"
I20260806 05:02:32.489514 7026 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260806 05:02:32.492703 7041 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 05:02:32.493953 7026 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 05:02:32.494143 7026 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785992227050689-9601-0/minicluster-data/master-0/data,/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785992227050689-9601-0/minicluster-data/master-0/wal
uuid: "76c1246e08154426aaf147c008a100e9"
format_stamp: "Formatted at 2026-08-06 05:02:32 on dist-test-slave-x4qh"
server_key: "3f0874df315ef1923699df5fb84e7697"
server_key_iv: "e127d88837259f1fabd258e75358fb88"
server_key_version: "encryptionkey@0"
I20260806 05:02:32.494360 7026 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785992227050689-9601-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785992227050689-9601-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785992227050689-9601-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 05:02:32.512041 7026 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 05:02:32.512854 7026 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 05:02:32.513139 7026 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 05:02:32.532136 7026 rpc_server.cc:307] RPC server started. Bound to: 127.9.96.126:35245
I20260806 05:02:32.532168 7124 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.96.126:35245 every 8 connection(s)
I20260806 05:02:32.533448 7026 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785992227050689-9601-0/minicluster-data/master-0/data/info.pb
I20260806 05:02:32.537207 7125 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 05:02:32.537525 9601 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu as pid 7026
I20260806 05:02:32.537701 9601 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785992227050689-9601-0/minicluster-data/master-0/wal/instance
I20260806 05:02:32.538190 9601 external_mini_cluster.cc:1428] Setting key 15225ef51b74dbb81cb3f57592645cbd
I20260806 05:02:32.552038 7125 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 76c1246e08154426aaf147c008a100e9: Bootstrap starting.
I20260806 05:02:32.554152 7131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:02:32.540095 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32894 (local address 127.9.96.126:35245)
0806 05:02:32.541248 (+ 1153us) server_negotiation.cc:206] Beginning negotiation
0806 05:02:32.541262 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0806 05:02:32.541323 (+ 61us) server_negotiation.cc:407] Connection header received
0806 05:02:32.544008 (+ 2685us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 05:02:32.544034 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 05:02:32.544648 (+ 614us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 05:02:32.545155 (+ 507us) server_negotiation.cc:226] Negotiated authn=SASL
0806 05:02:32.545676 (+ 521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:02:32.547592 (+ 1916us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:02:32.548961 (+ 1369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:02:32.550069 (+ 1108us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:02:32.550251 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:02:32.551351 (+ 1100us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 05:02:32.551364 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 05:02:32.551416 (+ 52us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 05:02:32.552143 (+ 727us) server_negotiation.cc:922] calling sasl_server_start()
0806 05:02:32.552242 (+ 99us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 05:02:32.552267 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 05:02:32.552396 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0806 05:02:32.552569 (+ 173us) server_negotiation.cc:299] Negotiation successful
0806 05:02:32.552886 (+ 317us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":618,"thread_start_us":356,"threads_started":1}
I20260806 05:02:32.556068 7125 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 76c1246e08154426aaf147c008a100e9: Neither blocks nor log segments found. Creating new log.
I20260806 05:02:32.557220 7125 log.cc:824] T 00000000000000000000000000000000 P 76c1246e08154426aaf147c008a100e9: Log is configured to *not* fsync() on all Append() calls
I20260806 05:02:32.559784 7125 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 76c1246e08154426aaf147c008a100e9: No bootstrap required, opened a new log
I20260806 05:02:32.567238 7125 raft_consensus.cc:348] T 00000000000000000000000000000000 P 76c1246e08154426aaf147c008a100e9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76c1246e08154426aaf147c008a100e9" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 35245 } }
I20260806 05:02:32.567647 7125 raft_consensus.cc:374] T 00000000000000000000000000000000 P 76c1246e08154426aaf147c008a100e9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 05:02:32.567768 7125 raft_consensus.cc:729] T 00000000000000000000000000000000 P 76c1246e08154426aaf147c008a100e9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 76c1246e08154426aaf147c008a100e9, State: Initialized, Role: FOLLOWER
I20260806 05:02:32.568360 7125 consensus_queue.cc:260] T 00000000000000000000000000000000 P 76c1246e08154426aaf147c008a100e9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76c1246e08154426aaf147c008a100e9" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 35245 } }
I20260806 05:02:32.568595 7125 raft_consensus.cc:388] T 00000000000000000000000000000000 P 76c1246e08154426aaf147c008a100e9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 05:02:32.568727 7125 raft_consensus.cc:482] T 00000000000000000000000000000000 P 76c1246e08154426aaf147c008a100e9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 05:02:32.568907 7125 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 76c1246e08154426aaf147c008a100e9 [term 0 FOLLOWER]: Advancing to term 1
I20260806 05:02:32.571097 7125 raft_consensus.cc:504] T 00000000000000000000000000000000 P 76c1246e08154426aaf147c008a100e9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76c1246e08154426aaf147c008a100e9" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 35245 } }
I20260806 05:02:32.571697 7125 leader_election.cc:302] T 00000000000000000000000000000000 P 76c1246e08154426aaf147c008a100e9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 76c1246e08154426aaf147c008a100e9; no voters:
I20260806 05:02:32.572296 7125 leader_election.cc:288] T 00000000000000000000000000000000 P 76c1246e08154426aaf147c008a100e9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 05:02:32.572479 7134 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 76c1246e08154426aaf147c008a100e9 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 05:02:32.573274 7134 raft_consensus.cc:686] T 00000000000000000000000000000000 P 76c1246e08154426aaf147c008a100e9 [term 1 LEADER]: Becoming Leader. State: Replica: 76c1246e08154426aaf147c008a100e9, State: Running, Role: LEADER
I20260806 05:02:32.573786 7134 consensus_queue.cc:237] T 00000000000000000000000000000000 P 76c1246e08154426aaf147c008a100e9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76c1246e08154426aaf147c008a100e9" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 35245 } }
I20260806 05:02:32.574304 7125 sys_catalog.cc:564] T 00000000000000000000000000000000 P 76c1246e08154426aaf147c008a100e9 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 05:02:32.576879 7136 sys_catalog.cc:455] T 00000000000000000000000000000000 P 76c1246e08154426aaf147c008a100e9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "76c1246e08154426aaf147c008a100e9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76c1246e08154426aaf147c008a100e9" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 35245 } } }
I20260806 05:02:32.577376 7136 sys_catalog.cc:458] T 00000000000000000000000000000000 P 76c1246e08154426aaf147c008a100e9 [sys.catalog]: This master's current role is: LEADER
I20260806 05:02:32.577966 7138 sys_catalog.cc:455] T 00000000000000000000000000000000 P 76c1246e08154426aaf147c008a100e9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 76c1246e08154426aaf147c008a100e9. Latest consensus state: current_term: 1 leader_uuid: "76c1246e08154426aaf147c008a100e9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76c1246e08154426aaf147c008a100e9" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 35245 } } }
I20260806 05:02:32.578328 7138 sys_catalog.cc:458] T 00000000000000000000000000000000 P 76c1246e08154426aaf147c008a100e9 [sys.catalog]: This master's current role is: LEADER
I20260806 05:02:32.579216 7145 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 05:02:32.586215 7145 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 05:02:32.595428 7145 catalog_manager.cc:1347] Generated new cluster ID: 870901f4614c4dbfb6ca57e0613923f4
I20260806 05:02:32.595556 7145 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 05:02:32.614562 7145 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 05:02:32.615931 7145 catalog_manager.cc:1504] Loading token signing keys...
I20260806 05:02:32.621995 7145 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 76c1246e08154426aaf147c008a100e9: Generated new TSK 0
I20260806 05:02:32.622805 7145 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 05:02:32.638526 9601 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
/tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785992227050689-9601-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785992227050689-9601-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785992227050689-9601-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785992227050689-9601-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.96.65:0
--local_ip_for_outbound_sockets=127.9.96.65
--webserver_interface=127.9.96.65
--webserver_port=0
--tserver_master_addrs=127.9.96.126:35245
--builtin_ntp_servers=127.9.96.84:36671
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260806 05:02:32.983124 7161 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 05:02:32.983456 7161 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 05:02:32.983609 7161 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 05:02:32.990734 7161 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 05:02:32.990823 7161 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 05:02:32.990957 7161 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.96.65
I20260806 05:02:33.000272 7161 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.96.84:36671
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785992227050689-9601-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785992227050689-9601-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.96.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785992227050689-9601-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.96.65
--webserver_port=0
--tserver_master_addrs=127.9.96.126:35245
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.96.65
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785992227050689-9601-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 03:56:12 UTC on ccycloud.quasar-guijbm.root.comops.site
build id 4569618
ASAN enabled
I20260806 05:02:33.001492 7161 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 05:02:33.003346 7161 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 05:02:33.017130 7175 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 05:02:33.017141 7173 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 05:02:33.017732 7171 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 05:02:33.018059 7161 server_base.cc:1034] running on GCE node
I20260806 05:02:33.018743 7161 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 05:02:33.019832 7161 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 05:02:33.021065 7161 hybrid_clock.cc:648] HybridClock initialized: now 1785992553021023 us; error 36 us; skew 500 ppm
I20260806 05:02:33.021416 7161 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 05:02:33.024031 7161 webserver.cc:460] Webserver started at http://127.9.96.65:45789/ using document root <none> and password file <none>
I20260806 05:02:33.024797 7161 fs_manager.cc:359] Metadata directory not provided
I20260806 05:02:33.024930 7161 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 05:02:33.025295 7161 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 05:02:33.028478 7161 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785992227050689-9601-0/minicluster-data/ts-0/data/instance:
uuid: "d80021392632451988b6b1e8f4261187"
format_stamp: "Formatted at 2026-08-06 05:02:33 on dist-test-slave-x4qh"
server_key: "36154048c7f84f4896adb7eef788b15e"
server_key_iv: "cf043b9ec607d75f6bea1421a7138fae"
server_key_version: "encryptionkey@0"
I20260806 05:02:33.029351 7161 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785992227050689-9601-0/minicluster-data/ts-0/wal/instance:
uuid: "d80021392632451988b6b1e8f4261187"
format_stamp: "Formatted at 2026-08-06 05:02:33 on dist-test-slave-x4qh"
server_key: "36154048c7f84f4896adb7eef788b15e"
server_key_iv: "cf043b9ec607d75f6bea1421a7138fae"
server_key_version: "encryptionkey@0"
I20260806 05:02:33.034777 7161 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.003s
I20260806 05:02:33.038442 7186 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 05:02:33.039834 7161 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 05:02:33.040016 7161 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785992227050689-9601-0/minicluster-data/ts-0/data,/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785992227050689-9601-0/minicluster-data/ts-0/wal
uuid: "d80021392632451988b6b1e8f4261187"
format_stamp: "Formatted at 2026-08-06 05:02:33 on dist-test-slave-x4qh"
server_key: "36154048c7f84f4896adb7eef788b15e"
server_key_iv: "cf043b9ec607d75f6bea1421a7138fae"
server_key_version: "encryptionkey@0"
I20260806 05:02:33.040262 7161 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785992227050689-9601-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785992227050689-9601-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785992227050689-9601-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 05:02:33.067873 7161 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 05:02:33.068904 7161 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 05:02:33.069217 7161 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 05:02:33.070223 7161 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 05:02:33.071798 7161 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 05:02:33.071918 7161 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 05:02:33.072045 7161 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 05:02:33.072135 7161 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 05:02:33.106009 7161 rpc_server.cc:307] RPC server started. Bound to: 127.9.96.65:38463
I20260806 05:02:33.106096 7349 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.96.65:38463 every 8 connection(s)
I20260806 05:02:33.107328 7161 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785992227050689-9601-0/minicluster-data/ts-0/data/info.pb
I20260806 05:02:33.114310 9601 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu as pid 7161
I20260806 05:02:33.114482 9601 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785992227050689-9601-0/minicluster-data/ts-0/wal/instance
I20260806 05:02:33.114954 9601 external_mini_cluster.cc:1428] Setting key 1c3f6a62edd26562bc879dc4dda29b74
I20260806 05:02:33.117844 9601 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
/tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785992227050689-9601-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785992227050689-9601-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785992227050689-9601-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785992227050689-9601-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.96.66:0
--local_ip_for_outbound_sockets=127.9.96.66
--webserver_interface=127.9.96.66
--webserver_port=0
--tserver_master_addrs=127.9.96.126:35245
--builtin_ntp_servers=127.9.96.84:36671
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260806 05:02:33.120764 7354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:02:33.110150 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.96.65:43357 (local address 127.9.96.126:35245)
0806 05:02:33.110586 (+ 436us) server_negotiation.cc:206] Beginning negotiation
0806 05:02:33.110590 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 05:02:33.111789 (+ 1199us) server_negotiation.cc:407] Connection header received
0806 05:02:33.113604 (+ 1815us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 05:02:33.113609 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 05:02:33.113696 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 05:02:33.113823 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0806 05:02:33.114963 (+ 1140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:02:33.115853 (+ 890us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:02:33.117610 (+ 1757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:02:33.118312 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:02:33.118462 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:02:33.119797 (+ 1335us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 05:02:33.119805 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 05:02:33.119813 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 05:02:33.119839 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0806 05:02:33.119897 (+ 58us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 05:02:33.119915 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 05:02:33.120013 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0806 05:02:33.120210 (+ 197us) server_negotiation.cc:299] Negotiation successful
0806 05:02:33.120316 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":325,"thread_start_us":225,"threads_started":1}
I20260806 05:02:33.121486 7358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:02:33.110718 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:35245 (local address 127.9.96.65:43357)
0806 05:02:33.111627 (+ 909us) negotiation.cc:107] Waiting for socket to connect
0806 05:02:33.111663 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0806 05:02:33.113340 (+ 1677us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 05:02:33.114055 (+ 715us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 05:02:33.114062 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 05:02:33.114150 (+ 88us) client_negotiation.cc:189] Negotiated authn=SASL
0806 05:02:33.114797 (+ 647us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 05:02:33.114815 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 05:02:33.116008 (+ 1193us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 05:02:33.116012 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 05:02:33.117459 (+ 1447us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 05:02:33.117472 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 05:02:33.118518 (+ 1046us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 05:02:33.118522 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 05:02:33.118723 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:02:33.119478 (+ 755us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 05:02:33.119480 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0806 05:02:33.119517 (+ 37us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 05:02:33.119595 (+ 78us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 05:02:33.120016 (+ 421us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 05:02:33.120024 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 05:02:33.120030 (+ 6us) client_negotiation.cc:769] Sending connection context
0806 05:02:33.120168 (+ 138us) client_negotiation.cc:240] Negotiation successful
0806 05:02:33.120442 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":436,"thread_start_us":251,"threads_started":1}
I20260806 05:02:33.123067 7350 heartbeater.cc:347] Connected to a master server at 127.9.96.126:35245
I20260806 05:02:33.123343 7350 heartbeater.cc:464] Registering TS with master...
I20260806 05:02:33.124192 7350 heartbeater.cc:511] Master 127.9.96.126:35245 requested a full tablet report, sending...
I20260806 05:02:33.126157 7068 ts_manager.cc:194] Registered new tserver with Master: d80021392632451988b6b1e8f4261187 (127.9.96.65:38463)
I20260806 05:02:33.127627 7068 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.9.96.65:43357
W20260806 05:02:33.447611 7360 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 05:02:33.447887 7360 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 05:02:33.448037 7360 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 05:02:33.455708 7360 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 05:02:33.455816 7360 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 05:02:33.456001 7360 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.96.66
I20260806 05:02:33.465289 7360 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.96.84:36671
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785992227050689-9601-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785992227050689-9601-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.96.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785992227050689-9601-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.96.66
--webserver_port=0
--tserver_master_addrs=127.9.96.126:35245
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.96.66
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785992227050689-9601-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 03:56:12 UTC on ccycloud.quasar-guijbm.root.comops.site
build id 4569618
ASAN enabled
I20260806 05:02:33.466578 7360 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 05:02:33.468528 7360 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 05:02:33.482110 7366 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 05:02:33.482678 7368 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 05:02:33.482962 7365 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 05:02:33.484025 7360 server_base.cc:1034] running on GCE node
I20260806 05:02:33.484615 7360 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 05:02:33.485498 7360 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 05:02:33.486722 7360 hybrid_clock.cc:648] HybridClock initialized: now 1785992553486685 us; error 42 us; skew 500 ppm
I20260806 05:02:33.487071 7360 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 05:02:33.489321 7360 webserver.cc:460] Webserver started at http://127.9.96.66:42213/ using document root <none> and password file <none>
I20260806 05:02:33.489992 7360 fs_manager.cc:359] Metadata directory not provided
I20260806 05:02:33.490119 7360 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 05:02:33.490483 7360 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 05:02:33.493269 7360 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785992227050689-9601-0/minicluster-data/ts-1/data/instance:
uuid: "312e91ec0e8f4b71a9234c56f523d610"
format_stamp: "Formatted at 2026-08-06 05:02:33 on dist-test-slave-x4qh"
server_key: "3eb11371eadfae36a4839923ba421671"
server_key_iv: "fd0d9a7b4a52b5005b7b9663d845daa7"
server_key_version: "encryptionkey@0"
I20260806 05:02:33.494052 7360 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785992227050689-9601-0/minicluster-data/ts-1/wal/instance:
uuid: "312e91ec0e8f4b71a9234c56f523d610"
format_stamp: "Formatted at 2026-08-06 05:02:33 on dist-test-slave-x4qh"
server_key: "3eb11371eadfae36a4839923ba421671"
server_key_iv: "fd0d9a7b4a52b5005b7b9663d845daa7"
server_key_version: "encryptionkey@0"
I20260806 05:02:33.499236 7360 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260806 05:02:33.502630 7378 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 05:02:33.503868 7360 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 05:02:33.504029 7360 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785992227050689-9601-0/minicluster-data/ts-1/data,/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785992227050689-9601-0/minicluster-data/ts-1/wal
uuid: "312e91ec0e8f4b71a9234c56f523d610"
format_stamp: "Formatted at 2026-08-06 05:02:33 on dist-test-slave-x4qh"
server_key: "3eb11371eadfae36a4839923ba421671"
server_key_iv: "fd0d9a7b4a52b5005b7b9663d845daa7"
server_key_version: "encryptionkey@0"
I20260806 05:02:33.504230 7360 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785992227050689-9601-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785992227050689-9601-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785992227050689-9601-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 05:02:33.523723 7360 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 05:02:33.524484 7360 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 05:02:33.524770 7360 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 05:02:33.525681 7360 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 05:02:33.527134 7360 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 05:02:33.527254 7360 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 05:02:33.527408 7360 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 05:02:33.527509 7360 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.001s
I20260806 05:02:33.561664 7360 rpc_server.cc:307] RPC server started. Bound to: 127.9.96.66:33365
I20260806 05:02:33.561774 7548 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.96.66:33365 every 8 connection(s)
I20260806 05:02:33.562981 7360 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785992227050689-9601-0/minicluster-data/ts-1/data/info.pb
I20260806 05:02:33.568436 9601 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu as pid 7360
I20260806 05:02:33.568640 9601 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785992227050689-9601-0/minicluster-data/ts-1/wal/instance
I20260806 05:02:33.569222 9601 external_mini_cluster.cc:1428] Setting key 149b395bc0f5841c8ea9b30990683c5b
I20260806 05:02:33.572399 9601 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
/tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785992227050689-9601-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785992227050689-9601-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785992227050689-9601-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785992227050689-9601-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.96.67:0
--local_ip_for_outbound_sockets=127.9.96.67
--webserver_interface=127.9.96.67
--webserver_port=0
--tserver_master_addrs=127.9.96.126:35245
--builtin_ntp_servers=127.9.96.84:36671
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260806 05:02:33.577678 7354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:02:33.565596 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.96.66:38031 (local address 127.9.96.126:35245)
0806 05:02:33.565816 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0806 05:02:33.565822 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 05:02:33.567155 (+ 1333us) server_negotiation.cc:407] Connection header received
0806 05:02:33.569474 (+ 2319us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 05:02:33.569478 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 05:02:33.569544 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 05:02:33.569631 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0806 05:02:33.570721 (+ 1090us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:02:33.571730 (+ 1009us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:02:33.574317 (+ 2587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:02:33.574964 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:02:33.575130 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:02:33.576780 (+ 1650us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 05:02:33.576789 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 05:02:33.576797 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 05:02:33.576822 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0806 05:02:33.576866 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 05:02:33.576884 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 05:02:33.576985 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0806 05:02:33.577186 (+ 201us) server_negotiation.cc:299] Negotiation successful
0806 05:02:33.577296 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260806 05:02:33.578502 7551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:02:33.566087 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:35245 (local address 127.9.96.66:38031)
0806 05:02:33.567007 (+ 920us) negotiation.cc:107] Waiting for socket to connect
0806 05:02:33.567051 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0806 05:02:33.569164 (+ 2113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 05:02:33.569908 (+ 744us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 05:02:33.569916 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 05:02:33.569998 (+ 82us) client_negotiation.cc:189] Negotiated authn=SASL
0806 05:02:33.570563 (+ 565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 05:02:33.570579 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 05:02:33.571885 (+ 1306us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 05:02:33.571890 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 05:02:33.574153 (+ 2263us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 05:02:33.574173 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 05:02:33.575524 (+ 1351us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 05:02:33.575531 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 05:02:33.575740 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:02:33.576455 (+ 715us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 05:02:33.576458 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0806 05:02:33.576498 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 05:02:33.576576 (+ 78us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 05:02:33.576991 (+ 415us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 05:02:33.576999 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 05:02:33.577005 (+ 6us) client_negotiation.cc:769] Sending connection context
0806 05:02:33.577143 (+ 138us) client_negotiation.cc:240] Negotiation successful
0806 05:02:33.577438 (+ 295us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":466,"thread_start_us":276,"threads_started":1}
I20260806 05:02:33.579926 7549 heartbeater.cc:347] Connected to a master server at 127.9.96.126:35245
I20260806 05:02:33.580227 7549 heartbeater.cc:464] Registering TS with master...
I20260806 05:02:33.581013 7549 heartbeater.cc:511] Master 127.9.96.126:35245 requested a full tablet report, sending...
I20260806 05:02:33.582405 7068 ts_manager.cc:194] Registered new tserver with Master: 312e91ec0e8f4b71a9234c56f523d610 (127.9.96.66:33365)
I20260806 05:02:33.583215 7068 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.9.96.66:38031
W20260806 05:02:33.910104 7554 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 05:02:33.910368 7554 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 05:02:33.910548 7554 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 05:02:33.917742 7554 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 05:02:33.917831 7554 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 05:02:33.918008 7554 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.96.67
I20260806 05:02:33.926957 7554 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.96.84:36671
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785992227050689-9601-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785992227050689-9601-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.96.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785992227050689-9601-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.96.67
--webserver_port=0
--tserver_master_addrs=127.9.96.126:35245
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.96.67
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785992227050689-9601-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 03:56:12 UTC on ccycloud.quasar-guijbm.root.comops.site
build id 4569618
ASAN enabled
I20260806 05:02:33.928256 7554 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 05:02:33.930020 7554 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 05:02:33.943418 7564 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 05:02:33.943478 7563 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 05:02:33.943470 7567 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 05:02:33.944563 7554 server_base.cc:1034] running on GCE node
I20260806 05:02:33.945274 7554 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 05:02:33.946216 7554 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 05:02:33.947474 7554 hybrid_clock.cc:648] HybridClock initialized: now 1785992553947421 us; error 35 us; skew 500 ppm
I20260806 05:02:33.947801 7554 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 05:02:33.950198 7554 webserver.cc:460] Webserver started at http://127.9.96.67:46583/ using document root <none> and password file <none>
I20260806 05:02:33.950924 7554 fs_manager.cc:359] Metadata directory not provided
I20260806 05:02:33.951021 7554 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 05:02:33.951431 7554 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 05:02:33.954308 7554 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785992227050689-9601-0/minicluster-data/ts-2/data/instance:
uuid: "9de8351840a547969e380757bf7d5637"
format_stamp: "Formatted at 2026-08-06 05:02:33 on dist-test-slave-x4qh"
server_key: "372b25b352f1b7f3c49865eea3cc8190"
server_key_iv: "d6c6c99da571663176e146f74d349ec8"
server_key_version: "encryptionkey@0"
I20260806 05:02:33.955106 7554 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785992227050689-9601-0/minicluster-data/ts-2/wal/instance:
uuid: "9de8351840a547969e380757bf7d5637"
format_stamp: "Formatted at 2026-08-06 05:02:33 on dist-test-slave-x4qh"
server_key: "372b25b352f1b7f3c49865eea3cc8190"
server_key_iv: "d6c6c99da571663176e146f74d349ec8"
server_key_version: "encryptionkey@0"
I20260806 05:02:33.960294 7554 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.003s
I20260806 05:02:33.963712 7576 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 05:02:33.965016 7554 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260806 05:02:33.965189 7554 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785992227050689-9601-0/minicluster-data/ts-2/data,/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785992227050689-9601-0/minicluster-data/ts-2/wal
uuid: "9de8351840a547969e380757bf7d5637"
format_stamp: "Formatted at 2026-08-06 05:02:33 on dist-test-slave-x4qh"
server_key: "372b25b352f1b7f3c49865eea3cc8190"
server_key_iv: "d6c6c99da571663176e146f74d349ec8"
server_key_version: "encryptionkey@0"
I20260806 05:02:33.965405 7554 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785992227050689-9601-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785992227050689-9601-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785992227050689-9601-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 05:02:33.977423 7554 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 05:02:33.978212 7554 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 05:02:33.978511 7554 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 05:02:33.979511 7554 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 05:02:33.981120 7554 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 05:02:33.981244 7554 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 05:02:33.981372 7554 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 05:02:33.981457 7554 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 05:02:34.014494 7554 rpc_server.cc:307] RPC server started. Bound to: 127.9.96.67:40903
I20260806 05:02:34.014550 7750 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.96.67:40903 every 8 connection(s)
I20260806 05:02:34.015865 7554 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785992227050689-9601-0/minicluster-data/ts-2/data/info.pb
I20260806 05:02:34.023782 9601 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu as pid 7554
I20260806 05:02:34.023954 9601 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785992227050689-9601-0/minicluster-data/ts-2/wal/instance
I20260806 05:02:34.024479 9601 external_mini_cluster.cc:1428] Setting key 1d010f9978db9dd9eeb24fc489e6abba
I20260806 05:02:34.028818 7354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:02:34.018332 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.96.67:38843 (local address 127.9.96.126:35245)
0806 05:02:34.018522 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0806 05:02:34.018526 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 05:02:34.019940 (+ 1414us) server_negotiation.cc:407] Connection header received
0806 05:02:34.021758 (+ 1818us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 05:02:34.021764 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 05:02:34.021850 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 05:02:34.021959 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0806 05:02:34.022986 (+ 1027us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:02:34.023880 (+ 894us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:02:34.025866 (+ 1986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:02:34.026501 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:02:34.026604 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:02:34.028137 (+ 1533us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 05:02:34.028141 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 05:02:34.028145 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 05:02:34.028162 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0806 05:02:34.028195 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 05:02:34.028208 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 05:02:34.028284 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0806 05:02:34.028473 (+ 189us) server_negotiation.cc:299] Negotiation successful
0806 05:02:34.028560 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260806 05:02:34.029737 7753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:02:34.018876 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:35245 (local address 127.9.96.67:38843)
0806 05:02:34.019788 (+ 912us) negotiation.cc:107] Waiting for socket to connect
0806 05:02:34.019823 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0806 05:02:34.021483 (+ 1660us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 05:02:34.022185 (+ 702us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 05:02:34.022193 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 05:02:34.022286 (+ 93us) client_negotiation.cc:189] Negotiated authn=SASL
0806 05:02:34.022821 (+ 535us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 05:02:34.022838 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 05:02:34.024053 (+ 1215us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 05:02:34.024060 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 05:02:34.025688 (+ 1628us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 05:02:34.025706 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 05:02:34.026719 (+ 1013us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 05:02:34.026730 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 05:02:34.027014 (+ 284us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:02:34.027849 (+ 835us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 05:02:34.027852 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0806 05:02:34.027895 (+ 43us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 05:02:34.027965 (+ 70us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 05:02:34.028298 (+ 333us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 05:02:34.028306 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 05:02:34.028311 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 05:02:34.028451 (+ 140us) client_negotiation.cc:240] Negotiation successful
0806 05:02:34.028720 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":460,"thread_start_us":273,"threads_started":1}
I20260806 05:02:34.031039 7751 heartbeater.cc:347] Connected to a master server at 127.9.96.126:35245
I20260806 05:02:34.031340 7751 heartbeater.cc:464] Registering TS with master...
I20260806 05:02:34.032155 7751 heartbeater.cc:511] Master 127.9.96.126:35245 requested a full tablet report, sending...
I20260806 05:02:34.033455 7068 ts_manager.cc:194] Registered new tserver with Master: 9de8351840a547969e380757bf7d5637 (127.9.96.67:40903)
I20260806 05:02:34.034201 7068 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.9.96.67:38843
I20260806 05:02:34.039781 9601 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 05:02:34.049870 7354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:02:34.043833 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32896 (local address 127.9.96.126:35245)
0806 05:02:34.044052 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0806 05:02:34.044060 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 05:02:34.044485 (+ 425us) server_negotiation.cc:407] Connection header received
0806 05:02:34.044676 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 05:02:34.044681 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 05:02:34.044773 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 05:02:34.044878 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0806 05:02:34.045408 (+ 530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:02:34.046112 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:02:34.047470 (+ 1358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:02:34.047881 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:02:34.048013 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:02:34.049128 (+ 1115us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 05:02:34.049134 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 05:02:34.049139 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 05:02:34.049164 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0806 05:02:34.049196 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 05:02:34.049208 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 05:02:34.049299 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0806 05:02:34.049464 (+ 165us) server_negotiation.cc:299] Negotiation successful
0806 05:02:34.049552 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260806 05:02:34.053514 7068 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:32896:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 05:02:34.060299 7068 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 05:02:34.086047 7771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:02:34.072379 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.67:40903 (local address 127.0.0.1:44100)
0806 05:02:34.074048 (+ 1669us) negotiation.cc:107] Waiting for socket to connect
0806 05:02:34.074097 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0806 05:02:34.074346 (+ 249us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 05:02:34.076062 (+ 1716us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 05:02:34.076073 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 05:02:34.076118 (+ 45us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 05:02:34.076600 (+ 482us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 05:02:34.076617 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 05:02:34.080113 (+ 3496us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 05:02:34.080118 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 05:02:34.082732 (+ 2614us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 05:02:34.082749 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 05:02:34.085022 (+ 2273us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 05:02:34.085030 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 05:02:34.085247 (+ 217us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:02:34.085310 (+ 63us) client_negotiation.cc:769] Sending connection context
0806 05:02:34.085454 (+ 144us) client_negotiation.cc:240] Negotiation successful
0806 05:02:34.085675 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1294,"thread_start_us":7620,"threads_started":1}
I20260806 05:02:34.086099 7769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:02:34.072658 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44100 (local address 127.9.96.67:40903)
0806 05:02:34.075186 (+ 2528us) server_negotiation.cc:206] Beginning negotiation
0806 05:02:34.075198 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 05:02:34.075241 (+ 43us) server_negotiation.cc:407] Connection header received
0806 05:02:34.075405 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 05:02:34.075417 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 05:02:34.075768 (+ 351us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 05:02:34.076857 (+ 1089us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 05:02:34.077201 (+ 344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:02:34.079312 (+ 2111us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:02:34.082968 (+ 3656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:02:34.084546 (+ 1578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:02:34.084681 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:02:34.084957 (+ 276us) server_negotiation.cc:1035] Waiting for connection context
0806 05:02:34.085526 (+ 569us) server_negotiation.cc:299] Negotiation successful
0806 05:02:34.085684 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2275,"thread_start_us":2106,"threads_started":1}
I20260806 05:02:34.089330 7772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:02:34.073465 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39086 (local address 127.9.96.66:33365)
0806 05:02:34.079989 (+ 6524us) server_negotiation.cc:206] Beginning negotiation
0806 05:02:34.080002 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 05:02:34.080580 (+ 578us) server_negotiation.cc:407] Connection header received
0806 05:02:34.080789 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 05:02:34.080798 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 05:02:34.081030 (+ 232us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 05:02:34.081247 (+ 217us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 05:02:34.082211 (+ 964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:02:34.083572 (+ 1361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:02:34.086652 (+ 3080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:02:34.087981 (+ 1329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:02:34.088158 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:02:34.088385 (+ 227us) server_negotiation.cc:1035] Waiting for connection context
0806 05:02:34.088818 (+ 433us) server_negotiation.cc:299] Negotiation successful
0806 05:02:34.088994 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6318,"thread_start_us":6163,"threads_started":1}
I20260806 05:02:34.090070 7767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:02:34.072828 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.66:33365 (local address 127.0.0.1:39086)
0806 05:02:34.080315 (+ 7487us) negotiation.cc:107] Waiting for socket to connect
0806 05:02:34.080334 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0806 05:02:34.080469 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 05:02:34.081440 (+ 971us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 05:02:34.081447 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 05:02:34.081484 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 05:02:34.082000 (+ 516us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 05:02:34.082013 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 05:02:34.083854 (+ 1841us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 05:02:34.083859 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 05:02:34.086076 (+ 2217us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 05:02:34.086094 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 05:02:34.088182 (+ 2088us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 05:02:34.088188 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 05:02:34.088387 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:02:34.088426 (+ 39us) client_negotiation.cc:769] Sending connection context
0806 05:02:34.089528 (+ 1102us) client_negotiation.cc:240] Negotiation successful
0806 05:02:34.089689 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7343,"mutex_wait_us":61,"thread_start_us":770,"threads_started":1}
I20260806 05:02:34.090348 7773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:02:34.080771 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55676 (local address 127.9.96.65:38463)
0806 05:02:34.081325 (+ 554us) server_negotiation.cc:206] Beginning negotiation
0806 05:02:34.081333 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 05:02:34.082222 (+ 889us) server_negotiation.cc:407] Connection header received
0806 05:02:34.082360 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 05:02:34.082372 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 05:02:34.082526 (+ 154us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 05:02:34.082707 (+ 181us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 05:02:34.083328 (+ 621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:02:34.084779 (+ 1451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:02:34.087505 (+ 2726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:02:34.088899 (+ 1394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:02:34.089034 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:02:34.089271 (+ 237us) server_negotiation.cc:1035] Waiting for connection context
0806 05:02:34.089778 (+ 507us) server_negotiation.cc:299] Negotiation successful
0806 05:02:34.089948 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":398,"thread_start_us":299,"threads_started":1}
I20260806 05:02:34.090368 7774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:02:34.081408 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.65:38463 (local address 127.0.0.1:55676)
0806 05:02:34.082080 (+ 672us) negotiation.cc:107] Waiting for socket to connect
0806 05:02:34.082099 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0806 05:02:34.082211 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 05:02:34.082697 (+ 486us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 05:02:34.082701 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 05:02:34.082726 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 05:02:34.083151 (+ 425us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 05:02:34.083164 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 05:02:34.084917 (+ 1753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 05:02:34.084925 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 05:02:34.087286 (+ 2361us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 05:02:34.087301 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 05:02:34.089410 (+ 2109us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 05:02:34.089416 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 05:02:34.089583 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:02:34.089614 (+ 31us) client_negotiation.cc:769] Sending connection context
0806 05:02:34.089707 (+ 93us) client_negotiation.cc:240] Negotiation successful
0806 05:02:34.089927 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":536,"mutex_wait_us":3,"thread_start_us":335,"threads_started":1}
I20260806 05:02:34.091717 7654 tablet_service.cc:1467] Processing CreateTablet for tablet 79c4da79210942a99b34de2b7681527f (DEFAULT_TABLE table=test-table [id=0965fdedeff04a11b5b273fe6a4b03bc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 05:02:34.093444 7654 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 79c4da79210942a99b34de2b7681527f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 05:02:34.093354 7252 tablet_service.cc:1467] Processing CreateTablet for tablet 79c4da79210942a99b34de2b7681527f (DEFAULT_TABLE table=test-table [id=0965fdedeff04a11b5b273fe6a4b03bc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 05:02:34.093813 7456 tablet_service.cc:1467] Processing CreateTablet for tablet 79c4da79210942a99b34de2b7681527f (DEFAULT_TABLE table=test-table [id=0965fdedeff04a11b5b273fe6a4b03bc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 05:02:34.094573 7252 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 79c4da79210942a99b34de2b7681527f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 05:02:34.095100 7456 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 79c4da79210942a99b34de2b7681527f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 05:02:34.114466 7777 tablet_bootstrap.cc:492] T 79c4da79210942a99b34de2b7681527f P 9de8351840a547969e380757bf7d5637: Bootstrap starting.
I20260806 05:02:34.121479 7777 tablet_bootstrap.cc:654] T 79c4da79210942a99b34de2b7681527f P 9de8351840a547969e380757bf7d5637: Neither blocks nor log segments found. Creating new log.
I20260806 05:02:34.122466 7778 tablet_bootstrap.cc:492] T 79c4da79210942a99b34de2b7681527f P 312e91ec0e8f4b71a9234c56f523d610: Bootstrap starting.
I20260806 05:02:34.122984 7775 tablet_bootstrap.cc:492] T 79c4da79210942a99b34de2b7681527f P d80021392632451988b6b1e8f4261187: Bootstrap starting.
I20260806 05:02:34.124769 7777 log.cc:824] T 79c4da79210942a99b34de2b7681527f P 9de8351840a547969e380757bf7d5637: Log is configured to *not* fsync() on all Append() calls
I20260806 05:02:34.126104 7778 tablet_bootstrap.cc:654] T 79c4da79210942a99b34de2b7681527f P 312e91ec0e8f4b71a9234c56f523d610: Neither blocks nor log segments found. Creating new log.
I20260806 05:02:34.126382 7775 tablet_bootstrap.cc:654] T 79c4da79210942a99b34de2b7681527f P d80021392632451988b6b1e8f4261187: Neither blocks nor log segments found. Creating new log.
I20260806 05:02:34.127121 7778 log.cc:824] T 79c4da79210942a99b34de2b7681527f P 312e91ec0e8f4b71a9234c56f523d610: Log is configured to *not* fsync() on all Append() calls
I20260806 05:02:34.127616 7775 log.cc:824] T 79c4da79210942a99b34de2b7681527f P d80021392632451988b6b1e8f4261187: Log is configured to *not* fsync() on all Append() calls
I20260806 05:02:34.128340 7777 tablet_bootstrap.cc:492] T 79c4da79210942a99b34de2b7681527f P 9de8351840a547969e380757bf7d5637: No bootstrap required, opened a new log
I20260806 05:02:34.128737 7777 ts_tablet_manager.cc:1397] T 79c4da79210942a99b34de2b7681527f P 9de8351840a547969e380757bf7d5637: Time spent bootstrapping tablet: real 0.015s user 0.009s sys 0.000s
I20260806 05:02:34.130242 7350 heartbeater.cc:503] Master 127.9.96.126:35245 was elected leader, sending a full tablet report...
I20260806 05:02:34.132045 7778 tablet_bootstrap.cc:492] T 79c4da79210942a99b34de2b7681527f P 312e91ec0e8f4b71a9234c56f523d610: No bootstrap required, opened a new log
I20260806 05:02:34.132300 7778 ts_tablet_manager.cc:1397] T 79c4da79210942a99b34de2b7681527f P 312e91ec0e8f4b71a9234c56f523d610: Time spent bootstrapping tablet: real 0.010s user 0.007s sys 0.000s
I20260806 05:02:34.138248 7775 tablet_bootstrap.cc:492] T 79c4da79210942a99b34de2b7681527f P d80021392632451988b6b1e8f4261187: No bootstrap required, opened a new log
I20260806 05:02:34.138693 7775 ts_tablet_manager.cc:1397] T 79c4da79210942a99b34de2b7681527f P d80021392632451988b6b1e8f4261187: Time spent bootstrapping tablet: real 0.016s user 0.003s sys 0.009s
I20260806 05:02:34.140806 7777 raft_consensus.cc:348] T 79c4da79210942a99b34de2b7681527f P 9de8351840a547969e380757bf7d5637 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9de8351840a547969e380757bf7d5637" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 40903 } } peers { permanent_uuid: "312e91ec0e8f4b71a9234c56f523d610" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 33365 } } peers { permanent_uuid: "d80021392632451988b6b1e8f4261187" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 38463 } }
I20260806 05:02:34.140966 7778 raft_consensus.cc:348] T 79c4da79210942a99b34de2b7681527f P 312e91ec0e8f4b71a9234c56f523d610 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9de8351840a547969e380757bf7d5637" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 40903 } } peers { permanent_uuid: "312e91ec0e8f4b71a9234c56f523d610" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 33365 } } peers { permanent_uuid: "d80021392632451988b6b1e8f4261187" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 38463 } }
I20260806 05:02:34.141294 7777 raft_consensus.cc:374] T 79c4da79210942a99b34de2b7681527f P 9de8351840a547969e380757bf7d5637 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 05:02:34.141386 7777 raft_consensus.cc:729] T 79c4da79210942a99b34de2b7681527f P 9de8351840a547969e380757bf7d5637 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9de8351840a547969e380757bf7d5637, State: Initialized, Role: FOLLOWER
I20260806 05:02:34.141417 7778 raft_consensus.cc:374] T 79c4da79210942a99b34de2b7681527f P 312e91ec0e8f4b71a9234c56f523d610 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 05:02:34.141521 7778 raft_consensus.cc:729] T 79c4da79210942a99b34de2b7681527f P 312e91ec0e8f4b71a9234c56f523d610 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 312e91ec0e8f4b71a9234c56f523d610, State: Initialized, Role: FOLLOWER
I20260806 05:02:34.142112 7777 consensus_queue.cc:260] T 79c4da79210942a99b34de2b7681527f P 9de8351840a547969e380757bf7d5637 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9de8351840a547969e380757bf7d5637" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 40903 } } peers { permanent_uuid: "312e91ec0e8f4b71a9234c56f523d610" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 33365 } } peers { permanent_uuid: "d80021392632451988b6b1e8f4261187" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 38463 } }
I20260806 05:02:34.142119 7778 consensus_queue.cc:260] T 79c4da79210942a99b34de2b7681527f P 312e91ec0e8f4b71a9234c56f523d610 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9de8351840a547969e380757bf7d5637" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 40903 } } peers { permanent_uuid: "312e91ec0e8f4b71a9234c56f523d610" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 33365 } } peers { permanent_uuid: "d80021392632451988b6b1e8f4261187" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 38463 } }
I20260806 05:02:34.143909 7549 heartbeater.cc:503] Master 127.9.96.126:35245 was elected leader, sending a full tablet report...
I20260806 05:02:34.143945 7777 ts_tablet_manager.cc:1428] T 79c4da79210942a99b34de2b7681527f P 9de8351840a547969e380757bf7d5637: Time spent starting tablet: real 0.015s user 0.015s sys 0.000s
I20260806 05:02:34.144523 7751 heartbeater.cc:503] Master 127.9.96.126:35245 was elected leader, sending a full tablet report...
I20260806 05:02:34.144564 7778 ts_tablet_manager.cc:1428] T 79c4da79210942a99b34de2b7681527f P 312e91ec0e8f4b71a9234c56f523d610: Time spent starting tablet: real 0.012s user 0.012s sys 0.000s
I20260806 05:02:34.148680 7775 raft_consensus.cc:348] T 79c4da79210942a99b34de2b7681527f P d80021392632451988b6b1e8f4261187 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9de8351840a547969e380757bf7d5637" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 40903 } } peers { permanent_uuid: "312e91ec0e8f4b71a9234c56f523d610" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 33365 } } peers { permanent_uuid: "d80021392632451988b6b1e8f4261187" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 38463 } }
I20260806 05:02:34.149194 7775 raft_consensus.cc:374] T 79c4da79210942a99b34de2b7681527f P d80021392632451988b6b1e8f4261187 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 05:02:34.149302 7775 raft_consensus.cc:729] T 79c4da79210942a99b34de2b7681527f P d80021392632451988b6b1e8f4261187 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d80021392632451988b6b1e8f4261187, State: Initialized, Role: FOLLOWER
I20260806 05:02:34.149820 7775 consensus_queue.cc:260] T 79c4da79210942a99b34de2b7681527f P d80021392632451988b6b1e8f4261187 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9de8351840a547969e380757bf7d5637" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 40903 } } peers { permanent_uuid: "312e91ec0e8f4b71a9234c56f523d610" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 33365 } } peers { permanent_uuid: "d80021392632451988b6b1e8f4261187" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 38463 } }
I20260806 05:02:34.151634 7775 ts_tablet_manager.cc:1428] T 79c4da79210942a99b34de2b7681527f P d80021392632451988b6b1e8f4261187: Time spent starting tablet: real 0.013s user 0.008s sys 0.007s
I20260806 05:02:34.180441 7786 raft_consensus.cc:482] T 79c4da79210942a99b34de2b7681527f P 312e91ec0e8f4b71a9234c56f523d610 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 05:02:34.180815 7786 raft_consensus.cc:504] T 79c4da79210942a99b34de2b7681527f P 312e91ec0e8f4b71a9234c56f523d610 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9de8351840a547969e380757bf7d5637" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 40903 } } peers { permanent_uuid: "312e91ec0e8f4b71a9234c56f523d610" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 33365 } } peers { permanent_uuid: "d80021392632451988b6b1e8f4261187" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 38463 } }
I20260806 05:02:34.182478 7786 leader_election.cc:288] T 79c4da79210942a99b34de2b7681527f P 312e91ec0e8f4b71a9234c56f523d610 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9de8351840a547969e380757bf7d5637 (127.9.96.67:40903), d80021392632451988b6b1e8f4261187 (127.9.96.65:38463)
I20260806 05:02:34.190236 7769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:02:34.182921 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.96.66:40857 (local address 127.9.96.67:40903)
0806 05:02:34.183168 (+ 247us) server_negotiation.cc:206] Beginning negotiation
0806 05:02:34.183174 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 05:02:34.183529 (+ 355us) server_negotiation.cc:407] Connection header received
0806 05:02:34.183809 (+ 280us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 05:02:34.183815 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 05:02:34.183897 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 05:02:34.183967 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 05:02:34.184708 (+ 741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:02:34.185722 (+ 1014us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:02:34.188024 (+ 2302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:02:34.189235 (+ 1211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:02:34.189382 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:02:34.189493 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0806 05:02:34.189841 (+ 348us) server_negotiation.cc:299] Negotiation successful
0806 05:02:34.189985 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260806 05:02:34.190246 7773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:02:34.183815 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.96.66:45697 (local address 127.9.96.65:38463)
0806 05:02:34.184058 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0806 05:02:34.184068 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 05:02:34.184521 (+ 453us) server_negotiation.cc:407] Connection header received
0806 05:02:34.184669 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 05:02:34.184677 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 05:02:34.184773 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 05:02:34.184846 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 05:02:34.185445 (+ 599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:02:34.186392 (+ 947us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:02:34.188296 (+ 1904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:02:34.189258 (+ 962us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:02:34.189432 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:02:34.189522 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0806 05:02:34.189875 (+ 353us) server_negotiation.cc:299] Negotiation successful
0806 05:02:34.189998 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260806 05:02:34.190472 7793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:02:34.182798 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.67:40903 (local address 127.9.96.66:40857)
0806 05:02:34.183421 (+ 623us) negotiation.cc:107] Waiting for socket to connect
0806 05:02:34.183443 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0806 05:02:34.183613 (+ 170us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 05:02:34.184122 (+ 509us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 05:02:34.184125 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 05:02:34.184150 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 05:02:34.184559 (+ 409us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 05:02:34.184576 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 05:02:34.185857 (+ 1281us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 05:02:34.185863 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 05:02:34.187839 (+ 1976us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 05:02:34.187852 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 05:02:34.189458 (+ 1606us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 05:02:34.189465 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 05:02:34.189638 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:02:34.189702 (+ 64us) client_negotiation.cc:769] Sending connection context
0806 05:02:34.189778 (+ 76us) client_negotiation.cc:240] Negotiation successful
0806 05:02:34.189939 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":455,"mutex_wait_us":50,"thread_start_us":299,"threads_started":1}
I20260806 05:02:34.190544 7794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:02:34.183530 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.65:38463 (local address 127.9.96.66:45697)
0806 05:02:34.184413 (+ 883us) negotiation.cc:107] Waiting for socket to connect
0806 05:02:34.184428 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 05:02:34.184545 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 05:02:34.184899 (+ 354us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 05:02:34.184903 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 05:02:34.184928 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 05:02:34.185280 (+ 352us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 05:02:34.185292 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 05:02:34.186535 (+ 1243us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 05:02:34.186539 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 05:02:34.188103 (+ 1564us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 05:02:34.188118 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 05:02:34.189506 (+ 1388us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 05:02:34.189515 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 05:02:34.189679 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:02:34.189719 (+ 40us) client_negotiation.cc:769] Sending connection context
0806 05:02:34.189830 (+ 111us) client_negotiation.cc:240] Negotiation successful
0806 05:02:34.189998 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":770,"mutex_wait_us":59,"thread_start_us":546,"threads_started":1}
I20260806 05:02:34.191820 7680 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "79c4da79210942a99b34de2b7681527f" candidate_uuid: "312e91ec0e8f4b71a9234c56f523d610" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9de8351840a547969e380757bf7d5637" is_pre_election: true
I20260806 05:02:34.191785 7283 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "79c4da79210942a99b34de2b7681527f" candidate_uuid: "312e91ec0e8f4b71a9234c56f523d610" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d80021392632451988b6b1e8f4261187" is_pre_election: true
I20260806 05:02:34.192422 7283 raft_consensus.cc:2454] T 79c4da79210942a99b34de2b7681527f P d80021392632451988b6b1e8f4261187 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 312e91ec0e8f4b71a9234c56f523d610 in term 0.
I20260806 05:02:34.192422 7680 raft_consensus.cc:2454] T 79c4da79210942a99b34de2b7681527f P 9de8351840a547969e380757bf7d5637 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 312e91ec0e8f4b71a9234c56f523d610 in term 0.
I20260806 05:02:34.193143 7379 leader_election.cc:302] T 79c4da79210942a99b34de2b7681527f P 312e91ec0e8f4b71a9234c56f523d610 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 312e91ec0e8f4b71a9234c56f523d610, 9de8351840a547969e380757bf7d5637; no voters:
I20260806 05:02:34.193527 7786 raft_consensus.cc:2781] T 79c4da79210942a99b34de2b7681527f P 312e91ec0e8f4b71a9234c56f523d610 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 05:02:34.193711 7786 raft_consensus.cc:482] T 79c4da79210942a99b34de2b7681527f P 312e91ec0e8f4b71a9234c56f523d610 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 05:02:34.193850 7786 raft_consensus.cc:3037] T 79c4da79210942a99b34de2b7681527f P 312e91ec0e8f4b71a9234c56f523d610 [term 0 FOLLOWER]: Advancing to term 1
I20260806 05:02:34.196240 7786 raft_consensus.cc:504] T 79c4da79210942a99b34de2b7681527f P 312e91ec0e8f4b71a9234c56f523d610 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9de8351840a547969e380757bf7d5637" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 40903 } } peers { permanent_uuid: "312e91ec0e8f4b71a9234c56f523d610" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 33365 } } peers { permanent_uuid: "d80021392632451988b6b1e8f4261187" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 38463 } }
I20260806 05:02:34.196825 7786 leader_election.cc:288] T 79c4da79210942a99b34de2b7681527f P 312e91ec0e8f4b71a9234c56f523d610 [CANDIDATE]: Term 1 election: Requested vote from peers 9de8351840a547969e380757bf7d5637 (127.9.96.67:40903), d80021392632451988b6b1e8f4261187 (127.9.96.65:38463)
I20260806 05:02:34.197397 7680 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "79c4da79210942a99b34de2b7681527f" candidate_uuid: "312e91ec0e8f4b71a9234c56f523d610" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9de8351840a547969e380757bf7d5637"
I20260806 05:02:34.197538 7283 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "79c4da79210942a99b34de2b7681527f" candidate_uuid: "312e91ec0e8f4b71a9234c56f523d610" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d80021392632451988b6b1e8f4261187"
I20260806 05:02:34.197636 7680 raft_consensus.cc:3037] T 79c4da79210942a99b34de2b7681527f P 9de8351840a547969e380757bf7d5637 [term 0 FOLLOWER]: Advancing to term 1
I20260806 05:02:34.197826 7283 raft_consensus.cc:3037] T 79c4da79210942a99b34de2b7681527f P d80021392632451988b6b1e8f4261187 [term 0 FOLLOWER]: Advancing to term 1
I20260806 05:02:34.201071 7283 raft_consensus.cc:2454] T 79c4da79210942a99b34de2b7681527f P d80021392632451988b6b1e8f4261187 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 312e91ec0e8f4b71a9234c56f523d610 in term 1.
I20260806 05:02:34.201071 7680 raft_consensus.cc:2454] T 79c4da79210942a99b34de2b7681527f P 9de8351840a547969e380757bf7d5637 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 312e91ec0e8f4b71a9234c56f523d610 in term 1.
I20260806 05:02:34.201472 7379 leader_election.cc:302] T 79c4da79210942a99b34de2b7681527f P 312e91ec0e8f4b71a9234c56f523d610 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 312e91ec0e8f4b71a9234c56f523d610, d80021392632451988b6b1e8f4261187; no voters:
I20260806 05:02:34.201874 7786 raft_consensus.cc:2781] T 79c4da79210942a99b34de2b7681527f P 312e91ec0e8f4b71a9234c56f523d610 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 05:02:34.202338 7786 raft_consensus.cc:686] T 79c4da79210942a99b34de2b7681527f P 312e91ec0e8f4b71a9234c56f523d610 [term 1 LEADER]: Becoming Leader. State: Replica: 312e91ec0e8f4b71a9234c56f523d610, State: Running, Role: LEADER
I20260806 05:02:34.202805 7786 consensus_queue.cc:237] T 79c4da79210942a99b34de2b7681527f P 312e91ec0e8f4b71a9234c56f523d610 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9de8351840a547969e380757bf7d5637" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 40903 } } peers { permanent_uuid: "312e91ec0e8f4b71a9234c56f523d610" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 33365 } } peers { permanent_uuid: "d80021392632451988b6b1e8f4261187" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 38463 } }
I20260806 05:02:34.207358 7064 catalog_manager.cc:5495] T 79c4da79210942a99b34de2b7681527f P 312e91ec0e8f4b71a9234c56f523d610 reported cstate change: term changed from 0 to 1, leader changed from <none> to 312e91ec0e8f4b71a9234c56f523d610 (127.9.96.66). New cstate: current_term: 1 leader_uuid: "312e91ec0e8f4b71a9234c56f523d610" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9de8351840a547969e380757bf7d5637" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 40903 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "312e91ec0e8f4b71a9234c56f523d610" member_type: VOTER last_known_addr { host: "127.9.96.66" port: 33365 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d80021392632451988b6b1e8f4261187" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 38463 } health_report { overall_health: UNKNOWN } } }
I20260806 05:02:34.236714 7772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:02:34.231232 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39088 (local address 127.9.96.66:33365)
0806 05:02:34.231482 (+ 250us) server_negotiation.cc:206] Beginning negotiation
0806 05:02:34.231488 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 05:02:34.231507 (+ 19us) server_negotiation.cc:407] Connection header received
0806 05:02:34.231733 (+ 226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 05:02:34.231737 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 05:02:34.231824 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 05:02:34.231977 (+ 153us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 05:02:34.232333 (+ 356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:02:34.233075 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:02:34.234522 (+ 1447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:02:34.234938 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:02:34.235065 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:02:34.235606 (+ 541us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 05:02:34.236108 (+ 502us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 05:02:34.236277 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0806 05:02:34.236358 (+ 81us) server_negotiation.cc:299] Negotiation successful
0806 05:02:34.236435 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260806 05:02:34.245460 7283 raft_consensus.cc:1261] T 79c4da79210942a99b34de2b7681527f P d80021392632451988b6b1e8f4261187 [term 1 FOLLOWER]: Refusing update from remote peer 312e91ec0e8f4b71a9234c56f523d610: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 05:02:34.245461 7680 raft_consensus.cc:1261] T 79c4da79210942a99b34de2b7681527f P 9de8351840a547969e380757bf7d5637 [term 1 FOLLOWER]: Refusing update from remote peer 312e91ec0e8f4b71a9234c56f523d610: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 05:02:34.246474 7796 consensus_queue.cc:1035] T 79c4da79210942a99b34de2b7681527f P 312e91ec0e8f4b71a9234c56f523d610 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9de8351840a547969e380757bf7d5637" member_type: VOTER last_known_addr { host: "127.9.96.67" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 05:02:34.246855 7786 consensus_queue.cc:1035] T 79c4da79210942a99b34de2b7681527f P 312e91ec0e8f4b71a9234c56f523d610 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d80021392632451988b6b1e8f4261187" member_type: VOTER last_known_addr { host: "127.9.96.65" port: 38463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 05:02:34.258481 7801 mvcc.cc:205] Tried to move back new op lower bound from 7315425502178017280 to 7315425502019018752. Current Snapshot: MvccSnapshot[applied={T|T < 7315425502178017280}]
I20260806 05:02:34.275338 7064 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:32896:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 05:02:34.275677 7064 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:32896:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 05:02:34.279407 7064 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 76c1246e08154426aaf147c008a100e9: Sending DeleteTablet for 3 replicas of tablet 79c4da79210942a99b34de2b7681527f
I20260806 05:02:34.280658 7654 tablet_service.cc:1514] Processing DeleteTablet for tablet 79c4da79210942a99b34de2b7681527f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 05:02:34 UTC) from {username='slave'} at 127.0.0.1:44100
I20260806 05:02:34.280679 7456 tablet_service.cc:1514] Processing DeleteTablet for tablet 79c4da79210942a99b34de2b7681527f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 05:02:34 UTC) from {username='slave'} at 127.0.0.1:39086
I20260806 05:02:34.281612 7252 tablet_service.cc:1514] Processing DeleteTablet for tablet 79c4da79210942a99b34de2b7681527f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 05:02:34 UTC) from {username='slave'} at 127.0.0.1:55676
I20260806 05:02:34.281714 9601 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu pid 7161
I20260806 05:02:34.301877 7815 tablet_replica.cc:330] T 79c4da79210942a99b34de2b7681527f P 312e91ec0e8f4b71a9234c56f523d610: stopping tablet replica
I20260806 05:02:34.302492 7815 raft_consensus.cc:2229] T 79c4da79210942a99b34de2b7681527f P 312e91ec0e8f4b71a9234c56f523d610 [term 1 LEADER]: Raft consensus shutting down.
I20260806 05:02:34.303292 7815 raft_consensus.cc:2258] T 79c4da79210942a99b34de2b7681527f P 312e91ec0e8f4b71a9234c56f523d610 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 05:02:34.305338 7816 tablet_replica.cc:330] T 79c4da79210942a99b34de2b7681527f P d80021392632451988b6b1e8f4261187: stopping tablet replica
I20260806 05:02:34.305761 7816 raft_consensus.cc:2229] T 79c4da79210942a99b34de2b7681527f P d80021392632451988b6b1e8f4261187 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 05:02:34.306182 7815 ts_tablet_manager.cc:1905] T 79c4da79210942a99b34de2b7681527f P 312e91ec0e8f4b71a9234c56f523d610: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 05:02:34.306272 7773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:02:34.283181 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55682 (local address 127.9.96.65:38463)
0806 05:02:34.283420 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0806 05:02:34.283428 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 05:02:34.293686 (+ 10258us) server_negotiation.cc:407] Connection header received
0806 05:02:34.294194 (+ 508us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 05:02:34.294203 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 05:02:34.294370 (+ 167us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 05:02:34.294461 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0806 05:02:34.295221 (+ 760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:02:34.296701 (+ 1480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:02:34.299260 (+ 2559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:02:34.300147 (+ 887us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:02:34.300265 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:02:34.304176 (+ 3911us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 05:02:34.304192 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 05:02:34.304211 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 05:02:34.305073 (+ 862us) server_negotiation.cc:922] calling sasl_server_start()
0806 05:02:34.305162 (+ 89us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 05:02:34.305194 (+ 32us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 05:02:34.305346 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0806 05:02:34.305683 (+ 337us) server_negotiation.cc:299] Negotiation successful
0806 05:02:34.305821 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260806 05:02:34.306885 7814 tablet_replica.cc:330] T 79c4da79210942a99b34de2b7681527f P 9de8351840a547969e380757bf7d5637: stopping tablet replica
I20260806 05:02:34.307312 7814 raft_consensus.cc:2229] T 79c4da79210942a99b34de2b7681527f P 9de8351840a547969e380757bf7d5637 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 05:02:34.307794 7344 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 05:02:34.307858 7814 raft_consensus.cc:2258] T 79c4da79210942a99b34de2b7681527f P 9de8351840a547969e380757bf7d5637 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 05:02:34.306296 7816 raft_consensus.cc:2258] T 79c4da79210942a99b34de2b7681527f P d80021392632451988b6b1e8f4261187 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 05:02:34.310231 7814 ts_tablet_manager.cc:1905] T 79c4da79210942a99b34de2b7681527f P 9de8351840a547969e380757bf7d5637: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 05:02:34.314432 7815 ts_tablet_manager.cc:1918] T 79c4da79210942a99b34de2b7681527f P 312e91ec0e8f4b71a9234c56f523d610: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 05:02:34.314597 7815 log.cc:1196] T 79c4da79210942a99b34de2b7681527f P 312e91ec0e8f4b71a9234c56f523d610: Deleting WAL directory at /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785992227050689-9601-0/minicluster-data/ts-1/wal/wals/79c4da79210942a99b34de2b7681527f
I20260806 05:02:34.315161 7815 ts_tablet_manager.cc:1939] T 79c4da79210942a99b34de2b7681527f P 312e91ec0e8f4b71a9234c56f523d610: Deleting consensus metadata
I20260806 05:02:34.317077 7814 ts_tablet_manager.cc:1918] T 79c4da79210942a99b34de2b7681527f P 9de8351840a547969e380757bf7d5637: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 05:02:34.317042 7046 catalog_manager.cc:4841] TS 312e91ec0e8f4b71a9234c56f523d610 (127.9.96.66:33365): tablet 79c4da79210942a99b34de2b7681527f (table test-table [id=0965fdedeff04a11b5b273fe6a4b03bc]) successfully deleted
I20260806 05:02:34.317238 7814 log.cc:1196] T 79c4da79210942a99b34de2b7681527f P 9de8351840a547969e380757bf7d5637: Deleting WAL directory at /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785992227050689-9601-0/minicluster-data/ts-2/wal/wals/79c4da79210942a99b34de2b7681527f
I20260806 05:02:34.317761 7814 ts_tablet_manager.cc:1939] T 79c4da79210942a99b34de2b7681527f P 9de8351840a547969e380757bf7d5637: Deleting consensus metadata
I20260806 05:02:34.319936 7042 catalog_manager.cc:4841] TS 9de8351840a547969e380757bf7d5637 (127.9.96.67:40903): tablet 79c4da79210942a99b34de2b7681527f (table test-table [id=0965fdedeff04a11b5b273fe6a4b03bc]) successfully deleted
I20260806 05:02:34.406659 9601 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu with pid 7161
I20260806 05:02:34.432197 9601 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu pid 7360
W20260806 05:02:34.432274 7042 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.9.96.65:38463 (error 108)
I20260806 05:02:34.434626 7774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:02:34.434061 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.65:38463 (local address 127.0.0.1:55698)
0806 05:02:34.434287 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0806 05:02:34.434429 (+ 142us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.96.65:38463: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
W20260806 05:02:34.434938 7042 catalog_manager.cc:4584] TS d80021392632451988b6b1e8f4261187 (127.9.96.65:38463): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 79c4da79210942a99b34de2b7681527f: Network error: Client connection negotiation failed: client connection to 127.9.96.65:38463: connect: Connection refused (error 111)
I20260806 05:02:34.444427 7772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:02:34.433311 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39096 (local address 127.9.96.66:33365)
0806 05:02:34.433509 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0806 05:02:34.433517 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 05:02:34.433648 (+ 131us) server_negotiation.cc:407] Connection header received
0806 05:02:34.433963 (+ 315us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 05:02:34.433968 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 05:02:34.434088 (+ 120us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 05:02:34.434203 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0806 05:02:34.434671 (+ 468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:02:34.435670 (+ 999us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:02:34.436918 (+ 1248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:02:34.437376 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:02:34.437527 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:02:34.442959 (+ 5432us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 05:02:34.442970 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 05:02:34.442983 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 05:02:34.443650 (+ 667us) server_negotiation.cc:922] calling sasl_server_start()
0806 05:02:34.443698 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 05:02:34.443724 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 05:02:34.443844 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0806 05:02:34.443989 (+ 145us) server_negotiation.cc:299] Negotiation successful
0806 05:02:34.444090 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260806 05:02:34.445134 7542 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 05:02:34.488490 7774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:02:34.487947 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.65:38463 (local address 127.0.0.1:55710)
0806 05:02:34.488173 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0806 05:02:34.488280 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.96.65:38463: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
I20260806 05:02:34.489559 7774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:02:34.489146 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.65:38463 (local address 127.0.0.1:55720)
0806 05:02:34.489316 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0806 05:02:34.489408 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.96.65:38463: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260806 05:02:34.550066 7774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:02:34.549594 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.65:38463 (local address 127.0.0.1:55722)
0806 05:02:34.549821 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0806 05:02:34.549904 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.96.65:38463: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
I20260806 05:02:34.551353 7774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:02:34.551000 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.65:38463 (local address 127.0.0.1:55728)
0806 05:02:34.551153 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0806 05:02:34.551233 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.96.65:38463: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260806 05:02:34.574195 9601 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu with pid 7360
I20260806 05:02:34.603597 9601 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu pid 7554
I20260806 05:02:34.610033 7769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:02:34.604353 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44102 (local address 127.9.96.67:40903)
0806 05:02:34.604576 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0806 05:02:34.604581 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 05:02:34.604600 (+ 19us) server_negotiation.cc:407] Connection header received
0806 05:02:34.604757 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 05:02:34.604761 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 05:02:34.604846 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 05:02:34.604933 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0806 05:02:34.605303 (+ 370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:02:34.606037 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:02:34.607286 (+ 1249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:02:34.607762 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:02:34.607859 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:02:34.608786 (+ 927us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 05:02:34.608795 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 05:02:34.608810 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 05:02:34.609367 (+ 557us) server_negotiation.cc:922] calling sasl_server_start()
0806 05:02:34.609414 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 05:02:34.609433 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 05:02:34.609529 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0806 05:02:34.609631 (+ 102us) server_negotiation.cc:299] Negotiation successful
0806 05:02:34.609713 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":95}
I20260806 05:02:34.610728 7738 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 05:02:34.632558 7774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:02:34.632108 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.65:38463 (local address 127.0.0.1:55740)
0806 05:02:34.632315 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0806 05:02:34.632396 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.96.65:38463: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
I20260806 05:02:34.633595 7774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:02:34.633212 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.65:38463 (local address 127.0.0.1:55754)
0806 05:02:34.633409 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0806 05:02:34.633484 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.96.65:38463: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":85}
I20260806 05:02:34.697829 9601 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu with pid 7554
I20260806 05:02:34.722499 9601 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu pid 7026
I20260806 05:02:34.723693 7119 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 05:02:34.830036 9601 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu with pid 7026
I20260806 05:02:34.830104 7774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:02:34.829488 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.65:38463 (local address 127.0.0.1:55768)
0806 05:02:34.829715 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0806 05:02:34.829877 (+ 162us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.96.65:38463: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
2026-08-06T05:02:34Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (2825 ms)
[----------] 6 tests from EncryptionPolicyTest (22971 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785992227050689-9601-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 05:02:34 dist-test-slave-x4qh krb5kdc[7829](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 05:02:34 dist-test-slave-x4qh krb5kdc[7829](info): set up 2 sockets
Aug 06 05:02:34 dist-test-slave-x4qh krb5kdc[7829](info): commencing operation
krb5kdc: starting...
W20260806 05:02:36.158265 9601 mini_kdc.cc:121] Time spent starting KDC: real 1.276s user 0.000s sys 0.040s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785992227050689-9601-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785992227050689-9601-0/krb5kdc/test-user.keytab.
Aug 06 05:02:36 dist-test-slave-x4qh krb5kdc[7829](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992556, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T05:02:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T05:02:36Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785992227050689-9601-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785992227050689-9601-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 05:02:36.442999 9601 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
/tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785992227050689-9601-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785992227050689-9601-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785992227050689-9601-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785992227050689-9601-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43503
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785992227050689-9601-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36973
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43503
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785992227050689-9601-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785992227050689-9601-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785992227050689-9601-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 05:02:36.775532 7844 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 05:02:36.775851 7844 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 05:02:36.776014 7844 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 05:02:36.783241 7844 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 05:02:36.783351 7844 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 05:02:36.783473 7844 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 05:02:36.783579 7844 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 05:02:36.783656 7844 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 05:02:36.793464 7844 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36973
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785992227050689-9601-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785992227050689-9601-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43503
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785992227050689-9601-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43503
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785992227050689-9601-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785992227050689-9601-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 03:56:12 UTC on ccycloud.quasar-guijbm.root.comops.site
build id 4569618
ASAN enabled
I20260806 05:02:36.794787 7844 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 05:02:36.796154 7844 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 05:02:36.808239 7849 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 05:02:36.808333 7850 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 05:02:36.808276 7854 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 05:02:36.809510 7844 server_base.cc:1034] running on GCE node
I20260806 05:02:36.810281 7844 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 05:02:36.811750 7844 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 05:02:36.812981 7844 hybrid_clock.cc:648] HybridClock initialized: now 1785992556812941 us; error 48 us; skew 500 ppm
I20260806 05:02:36.813335 7844 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 05:02:36 dist-test-slave-x4qh krb5kdc[7829](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992556, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 05:02:36.816645 7844 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 05:02:36.818248 7844 webserver.cc:460] Webserver started at http://127.0.0.1:45497/ using document root <none> and password file <none>
I20260806 05:02:36.818935 7844 fs_manager.cc:359] Metadata directory not provided
I20260806 05:02:36.819082 7844 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 05:02:36.819427 7844 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 05:02:36.822472 7844 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785992227050689-9601-0/minicluster-data/master-0/data/instance:
uuid: "8da06d03942a43ac9d6666a9f6523aa6"
format_stamp: "Formatted at 2026-08-06 05:02:36 on dist-test-slave-x4qh"
server_key: "f5f77ad2dadde748bf8f8131dc74c9d6"
server_key_iv: "3af337481f672acedf29219fbaabde42"
server_key_version: "encryptionkey@0"
I20260806 05:02:36.823266 7844 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785992227050689-9601-0/minicluster-data/master-0/wal/instance:
uuid: "8da06d03942a43ac9d6666a9f6523aa6"
format_stamp: "Formatted at 2026-08-06 05:02:36 on dist-test-slave-x4qh"
server_key: "f5f77ad2dadde748bf8f8131dc74c9d6"
server_key_iv: "3af337481f672acedf29219fbaabde42"
server_key_version: "encryptionkey@0"
I20260806 05:02:36.828704 7844 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260806 05:02:36.832252 7869 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 05:02:36.833642 7844 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 05:02:36.833865 7844 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785992227050689-9601-0/minicluster-data/master-0/data,/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785992227050689-9601-0/minicluster-data/master-0/wal
uuid: "8da06d03942a43ac9d6666a9f6523aa6"
format_stamp: "Formatted at 2026-08-06 05:02:36 on dist-test-slave-x4qh"
server_key: "f5f77ad2dadde748bf8f8131dc74c9d6"
server_key_iv: "3af337481f672acedf29219fbaabde42"
server_key_version: "encryptionkey@0"
I20260806 05:02:36.834064 7844 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785992227050689-9601-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785992227050689-9601-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785992227050689-9601-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 05:02:36.848989 7844 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 05:02:36.852890 7844 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 05:02:36.853165 7844 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 05:02:36.871516 7844 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43503
I20260806 05:02:36.871536 7942 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43503 every 8 connection(s)
I20260806 05:02:36.872836 7844 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785992227050689-9601-0/minicluster-data/master-0/data/info.pb
I20260806 05:02:36.873404 9601 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu as pid 7844
I20260806 05:02:36.873624 9601 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785992227050689-9601-0/minicluster-data/master-0/wal/instance
I20260806 05:02:36.874456 9601 external_mini_cluster.cc:1428] Setting key dfdd50f8f0f7cd6295a5ab1bf65ee3fc
I20260806 05:02:36.877085 7943 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 06 05:02:36 dist-test-slave-x4qh krb5kdc[7829](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992556, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260806 05:02:36.890053 7943 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8da06d03942a43ac9d6666a9f6523aa6: Bootstrap starting.
I20260806 05:02:36.893441 7943 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8da06d03942a43ac9d6666a9f6523aa6: Neither blocks nor log segments found. Creating new log.
I20260806 05:02:36.894577 7943 log.cc:824] T 00000000000000000000000000000000 P 8da06d03942a43ac9d6666a9f6523aa6: Log is configured to *not* fsync() on all Append() calls
I20260806 05:02:36.898016 7943 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8da06d03942a43ac9d6666a9f6523aa6: No bootstrap required, opened a new log
I20260806 05:02:36.898914 7947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:02:36.876437 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43821 (local address 127.0.0.1:43503)
0806 05:02:36.877245 (+ 808us) server_negotiation.cc:206] Beginning negotiation
0806 05:02:36.877260 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 05:02:36.877321 (+ 61us) server_negotiation.cc:407] Connection header received
0806 05:02:36.879724 (+ 2403us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 05:02:36.879750 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 05:02:36.880234 (+ 484us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 05:02:36.880660 (+ 426us) server_negotiation.cc:226] Negotiated authn=SASL
0806 05:02:36.882031 (+ 1371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:02:36.884127 (+ 2096us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:02:36.885401 (+ 1274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:02:36.886459 (+ 1058us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:02:36.886637 (+ 178us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:02:36.891044 (+ 4407us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 05:02:36.891052 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 05:02:36.891065 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 05:02:36.891107 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0806 05:02:36.894375 (+ 3268us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 05:02:36.895437 (+ 1062us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 05:02:36.895445 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 05:02:36.895453 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0806 05:02:36.895695 (+ 242us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 05:02:36.896174 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 05:02:36.896178 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 05:02:36.896180 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 05:02:36.896679 (+ 499us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 05:02:36.896916 (+ 237us) server_negotiation.cc:1035] Waiting for connection context
0806 05:02:36.897430 (+ 514us) server_negotiation.cc:299] Negotiation successful
0806 05:02:36.897714 (+ 284us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":576,"thread_start_us":331,"threads_started":1}
I20260806 05:02:36.903124 9601 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 05:02:36.906632 7943 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8da06d03942a43ac9d6666a9f6523aa6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8da06d03942a43ac9d6666a9f6523aa6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43503 } }
I20260806 05:02:36.906991 7943 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8da06d03942a43ac9d6666a9f6523aa6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 05:02:36.907094 7943 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8da06d03942a43ac9d6666a9f6523aa6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8da06d03942a43ac9d6666a9f6523aa6, State: Initialized, Role: FOLLOWER
I20260806 05:02:36.907711 7943 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8da06d03942a43ac9d6666a9f6523aa6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8da06d03942a43ac9d6666a9f6523aa6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43503 } }
I20260806 05:02:36.907954 7943 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8da06d03942a43ac9d6666a9f6523aa6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 05:02:36.908155 7943 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8da06d03942a43ac9d6666a9f6523aa6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 05:02:36.908365 7943 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8da06d03942a43ac9d6666a9f6523aa6 [term 0 FOLLOWER]: Advancing to term 1
I20260806 05:02:36.911464 7943 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8da06d03942a43ac9d6666a9f6523aa6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8da06d03942a43ac9d6666a9f6523aa6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43503 } }
I20260806 05:02:36.912096 7943 leader_election.cc:302] T 00000000000000000000000000000000 P 8da06d03942a43ac9d6666a9f6523aa6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8da06d03942a43ac9d6666a9f6523aa6; no voters:
I20260806 05:02:36.912896 7943 leader_election.cc:288] T 00000000000000000000000000000000 P 8da06d03942a43ac9d6666a9f6523aa6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 05:02:36.913010 7956 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8da06d03942a43ac9d6666a9f6523aa6 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 05:02:36.914417 7956 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8da06d03942a43ac9d6666a9f6523aa6 [term 1 LEADER]: Becoming Leader. State: Replica: 8da06d03942a43ac9d6666a9f6523aa6, State: Running, Role: LEADER
I20260806 05:02:36.915091 7956 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8da06d03942a43ac9d6666a9f6523aa6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8da06d03942a43ac9d6666a9f6523aa6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43503 } }
I20260806 05:02:36.915529 7943 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8da06d03942a43ac9d6666a9f6523aa6 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 05:02:36.918182 7958 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8da06d03942a43ac9d6666a9f6523aa6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8da06d03942a43ac9d6666a9f6523aa6. Latest consensus state: current_term: 1 leader_uuid: "8da06d03942a43ac9d6666a9f6523aa6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8da06d03942a43ac9d6666a9f6523aa6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43503 } } }
I20260806 05:02:36.918633 7958 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8da06d03942a43ac9d6666a9f6523aa6 [sys.catalog]: This master's current role is: LEADER
I20260806 05:02:36.919435 7957 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8da06d03942a43ac9d6666a9f6523aa6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8da06d03942a43ac9d6666a9f6523aa6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8da06d03942a43ac9d6666a9f6523aa6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43503 } } }
I20260806 05:02:36.919800 7957 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8da06d03942a43ac9d6666a9f6523aa6 [sys.catalog]: This master's current role is: LEADER
I20260806 05:02:36.928527 7962 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 05:02:36.929239 7947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:02:36.909241 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48733 (local address 127.0.0.1:43503)
0806 05:02:36.909459 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0806 05:02:36.909469 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 05:02:36.909988 (+ 519us) server_negotiation.cc:407] Connection header received
0806 05:02:36.910151 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 05:02:36.910155 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 05:02:36.910215 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 05:02:36.910313 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0806 05:02:36.912237 (+ 1924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:02:36.913224 (+ 987us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:02:36.914960 (+ 1736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:02:36.915643 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:02:36.915775 (+ 132us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:02:36.918994 (+ 3219us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 05:02:36.919002 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 05:02:36.919008 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 05:02:36.919054 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0806 05:02:36.923798 (+ 4744us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 05:02:36.925126 (+ 1328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 05:02:36.925135 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 05:02:36.925140 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 05:02:36.925395 (+ 255us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 05:02:36.926132 (+ 737us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 05:02:36.926140 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 05:02:36.926145 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 05:02:36.926651 (+ 506us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 05:02:36.927050 (+ 399us) server_negotiation.cc:1035] Waiting for connection context
0806 05:02:36.928487 (+ 1437us) server_negotiation.cc:299] Negotiation successful
0806 05:02:36.928737 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260806 05:02:36.936884 7962 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 05:02:36.946254 7962 catalog_manager.cc:1347] Generated new cluster ID: c03f035bd0f04dadb00e119c25d8012a
I20260806 05:02:36.946381 7962 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 05:02:36.958855 7962 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 05:02:36.960704 7962 catalog_manager.cc:1504] Loading token signing keys...
I20260806 05:02:36.969651 7962 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8da06d03942a43ac9d6666a9f6523aa6: Generated new TSK 0
I20260806 05:02:36.970618 7962 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 05:02:36.979934 9601 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu pid 7844
I20260806 05:02:36.981927 7937 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 05:02:37.071689 9601 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu with pid 7844
W20260806 05:02:37.095003 7827 connection.cc:535] client connection to 127.0.0.1:43503 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:43503): Connection reset by peer (error 104)
2026-08-06T05:02:37Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (2232 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785992227050689-9601-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 05:02:37 dist-test-slave-x4qh krb5kdc[7992](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 05:02:37 dist-test-slave-x4qh krb5kdc[7992](info): set up 2 sockets
Aug 06 05:02:37 dist-test-slave-x4qh krb5kdc[7992](info): commencing operation
krb5kdc: starting...
W20260806 05:02:38.397388 9601 mini_kdc.cc:121] Time spent starting KDC: real 1.283s user 0.000s sys 0.038s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785992227050689-9601-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785992227050689-9601-0/krb5kdc/test-user.keytab.
Aug 06 05:02:38 dist-test-slave-x4qh krb5kdc[7992](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992558, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T05:02:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T05:02:38Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785992227050689-9601-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785992227050689-9601-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 05:02:38.674005 9601 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
/tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785992227050689-9601-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785992227050689-9601-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785992227050689-9601-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785992227050689-9601-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40267
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785992227050689-9601-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36113
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40267
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785992227050689-9601-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785992227050689-9601-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785992227050689-9601-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 05:02:39.006711 8015 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 05:02:39.007045 8015 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 05:02:39.007263 8015 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 05:02:39.015012 8015 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 05:02:39.015120 8015 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 05:02:39.015244 8015 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 05:02:39.015368 8015 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 05:02:39.015446 8015 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 05:02:39.026160 8015 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36113
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785992227050689-9601-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785992227050689-9601-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40267
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785992227050689-9601-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40267
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785992227050689-9601-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785992227050689-9601-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 03:56:12 UTC on ccycloud.quasar-guijbm.root.comops.site
build id 4569618
ASAN enabled
I20260806 05:02:39.027586 8015 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 05:02:39.028973 8015 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 05:02:39.042063 8024 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 05:02:39.042316 8027 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 05:02:39.042550 8023 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 05:02:39.043499 8015 server_base.cc:1034] running on GCE node
I20260806 05:02:39.044289 8015 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 05:02:39.045697 8015 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 05:02:39.046927 8015 hybrid_clock.cc:648] HybridClock initialized: now 1785992559046877 us; error 31 us; skew 500 ppm
I20260806 05:02:39.047307 8015 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 05:02:39 dist-test-slave-x4qh krb5kdc[7992](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992559, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 05:02:39.050942 8015 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 05:02:39.052529 8015 webserver.cc:460] Webserver started at http://127.0.0.1:33261/ using document root <none> and password file <none>
I20260806 05:02:39.053416 8015 fs_manager.cc:359] Metadata directory not provided
I20260806 05:02:39.053578 8015 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 05:02:39.053946 8015 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 05:02:39.057030 8015 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785992227050689-9601-0/minicluster-data/master-0/data/instance:
uuid: "8f082c1a856b42848c2f78de4a0b7b19"
format_stamp: "Formatted at 2026-08-06 05:02:39 on dist-test-slave-x4qh"
server_key: "f075293d55951c60a37bd56444ab5d99"
server_key_iv: "aa08dc0800cb164347e1995d9b66a22b"
server_key_version: "encryptionkey@0"
I20260806 05:02:39.057835 8015 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785992227050689-9601-0/minicluster-data/master-0/wal/instance:
uuid: "8f082c1a856b42848c2f78de4a0b7b19"
format_stamp: "Formatted at 2026-08-06 05:02:39 on dist-test-slave-x4qh"
server_key: "f075293d55951c60a37bd56444ab5d99"
server_key_iv: "aa08dc0800cb164347e1995d9b66a22b"
server_key_version: "encryptionkey@0"
I20260806 05:02:39.062973 8015 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.005s
I20260806 05:02:39.066448 8038 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 05:02:39.067791 8015 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 05:02:39.067982 8015 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785992227050689-9601-0/minicluster-data/master-0/data,/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785992227050689-9601-0/minicluster-data/master-0/wal
uuid: "8f082c1a856b42848c2f78de4a0b7b19"
format_stamp: "Formatted at 2026-08-06 05:02:39 on dist-test-slave-x4qh"
server_key: "f075293d55951c60a37bd56444ab5d99"
server_key_iv: "aa08dc0800cb164347e1995d9b66a22b"
server_key_version: "encryptionkey@0"
I20260806 05:02:39.068183 8015 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785992227050689-9601-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785992227050689-9601-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785992227050689-9601-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 05:02:39.085495 8015 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 05:02:39.090081 8015 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 05:02:39.090389 8015 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 05:02:39.109740 8015 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40267
I20260806 05:02:39.109790 8116 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40267 every 8 connection(s)
I20260806 05:02:39.111091 8015 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785992227050689-9601-0/minicluster-data/master-0/data/info.pb
I20260806 05:02:39.114809 8118 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 05:02:39.121441 9601 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu as pid 8015
I20260806 05:02:39.121657 9601 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785992227050689-9601-0/minicluster-data/master-0/wal/instance
I20260806 05:02:39.122107 9601 external_mini_cluster.cc:1428] Setting key da5f03177fbf364a8951ff4e6e8177b3
I20260806 05:02:39.125710 8118 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8f082c1a856b42848c2f78de4a0b7b19: Bootstrap starting.
I20260806 05:02:39.129284 8118 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8f082c1a856b42848c2f78de4a0b7b19: Neither blocks nor log segments found. Creating new log.
I20260806 05:02:39.130483 8118 log.cc:824] T 00000000000000000000000000000000 P 8f082c1a856b42848c2f78de4a0b7b19: Log is configured to *not* fsync() on all Append() calls
I20260806 05:02:39.133777 8118 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8f082c1a856b42848c2f78de4a0b7b19: No bootstrap required, opened a new log
I20260806 05:02:39.136499 8125 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:02:39.123924 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37409 (local address 127.0.0.1:40267)
0806 05:02:39.124738 (+ 814us) server_negotiation.cc:206] Beginning negotiation
0806 05:02:39.124750 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 05:02:39.124964 (+ 214us) server_negotiation.cc:407] Connection header received
0806 05:02:39.126929 (+ 1965us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 05:02:39.126951 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 05:02:39.127343 (+ 392us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 05:02:39.127760 (+ 417us) server_negotiation.cc:226] Negotiated authn=SASL
0806 05:02:39.128410 (+ 650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:02:39.129941 (+ 1531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:02:39.131994 (+ 2053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:02:39.132642 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:02:39.132792 (+ 150us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:02:39.134100 (+ 1308us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 05:02:39.134107 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 05:02:39.134119 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 05:02:39.134676 (+ 557us) server_negotiation.cc:922] calling sasl_server_start()
0806 05:02:39.134751 (+ 75us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 05:02:39.134778 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 05:02:39.134901 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0806 05:02:39.135235 (+ 334us) server_negotiation.cc:299] Negotiation successful
0806 05:02:39.135508 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":594,"thread_start_us":391,"threads_started":1}
I20260806 05:02:39.140303 9601 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 05:02:39.142477 8118 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8f082c1a856b42848c2f78de4a0b7b19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f082c1a856b42848c2f78de4a0b7b19" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40267 } }
I20260806 05:02:39.142932 8118 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8f082c1a856b42848c2f78de4a0b7b19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 05:02:39.143055 8118 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8f082c1a856b42848c2f78de4a0b7b19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f082c1a856b42848c2f78de4a0b7b19, State: Initialized, Role: FOLLOWER
I20260806 05:02:39.143728 8118 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8f082c1a856b42848c2f78de4a0b7b19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f082c1a856b42848c2f78de4a0b7b19" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40267 } }
I20260806 05:02:39.144053 8118 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8f082c1a856b42848c2f78de4a0b7b19 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 05:02:39.144217 8118 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8f082c1a856b42848c2f78de4a0b7b19 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 05:02:39.144443 8118 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8f082c1a856b42848c2f78de4a0b7b19 [term 0 FOLLOWER]: Advancing to term 1
I20260806 05:02:39.147622 8118 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8f082c1a856b42848c2f78de4a0b7b19 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f082c1a856b42848c2f78de4a0b7b19" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40267 } }
I20260806 05:02:39.148123 8118 leader_election.cc:302] T 00000000000000000000000000000000 P 8f082c1a856b42848c2f78de4a0b7b19 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8f082c1a856b42848c2f78de4a0b7b19; no voters:
I20260806 05:02:39.148808 8118 leader_election.cc:288] T 00000000000000000000000000000000 P 8f082c1a856b42848c2f78de4a0b7b19 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 05:02:39.148929 8134 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8f082c1a856b42848c2f78de4a0b7b19 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 05:02:39.151254 8134 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8f082c1a856b42848c2f78de4a0b7b19 [term 1 LEADER]: Becoming Leader. State: Replica: 8f082c1a856b42848c2f78de4a0b7b19, State: Running, Role: LEADER
I20260806 05:02:39.151734 8118 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8f082c1a856b42848c2f78de4a0b7b19 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 05:02:39.151866 8134 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8f082c1a856b42848c2f78de4a0b7b19 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f082c1a856b42848c2f78de4a0b7b19" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40267 } }
I20260806 05:02:39.154186 8125 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:02:39.145749 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51303 (local address 127.0.0.1:40267)
0806 05:02:39.145929 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0806 05:02:39.145935 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 05:02:39.146501 (+ 566us) server_negotiation.cc:407] Connection header received
0806 05:02:39.146662 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 05:02:39.146666 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 05:02:39.146718 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 05:02:39.146802 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0806 05:02:39.147359 (+ 557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:02:39.148194 (+ 835us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:02:39.151001 (+ 2807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:02:39.151696 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:02:39.151844 (+ 148us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:02:39.153073 (+ 1229us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 05:02:39.153078 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 05:02:39.153081 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 05:02:39.153098 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0806 05:02:39.153157 (+ 59us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 05:02:39.153173 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 05:02:39.153306 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0806 05:02:39.153731 (+ 425us) server_negotiation.cc:299] Negotiation successful
0806 05:02:39.153804 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260806 05:02:39.154712 8135 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8f082c1a856b42848c2f78de4a0b7b19 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8f082c1a856b42848c2f78de4a0b7b19" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f082c1a856b42848c2f78de4a0b7b19" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40267 } } }
I20260806 05:02:39.155164 8135 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8f082c1a856b42848c2f78de4a0b7b19 [sys.catalog]: This master's current role is: LEADER
I20260806 05:02:39.158816 8136 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8f082c1a856b42848c2f78de4a0b7b19 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8f082c1a856b42848c2f78de4a0b7b19. Latest consensus state: current_term: 1 leader_uuid: "8f082c1a856b42848c2f78de4a0b7b19" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f082c1a856b42848c2f78de4a0b7b19" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40267 } } }
I20260806 05:02:39.159215 8136 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8f082c1a856b42848c2f78de4a0b7b19 [sys.catalog]: This master's current role is: LEADER
I20260806 05:02:39.162907 8144 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 05:02:39.167304 8144 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 05:02:39.176198 8144 catalog_manager.cc:1347] Generated new cluster ID: 6e081399dc0d4666ba633e08af80d0c1
I20260806 05:02:39.176307 8144 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 05:02:39.212256 8144 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 05:02:39.214159 8144 catalog_manager.cc:1504] Loading token signing keys...
I20260806 05:02:39.232563 8144 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8f082c1a856b42848c2f78de4a0b7b19: Generated new TSK 0
I20260806 05:02:39.233538 8144 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 05:02:39.241729 9601 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu pid 8015
W20260806 05:02:39.242872 8106 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:37409
W20260806 05:02:39.243693 9601 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu pid 8015: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260806 05:02:39.243887 9601 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu with pid 8015
W20260806 05:02:39.273741 7990 connection.cc:535] client connection to 127.0.0.1:40267 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:40267): Connection reset by peer (error 104)
2026-08-06T05:02:39Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (2178 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785992227050689-9601-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 05:02:39 dist-test-slave-x4qh krb5kdc[8172](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 05:02:39 dist-test-slave-x4qh krb5kdc[8172](info): set up 2 sockets
Aug 06 05:02:39 dist-test-slave-x4qh krb5kdc[8172](info): commencing operation
krb5kdc: starting...
W20260806 05:02:40.554185 9601 mini_kdc.cc:121] Time spent starting KDC: real 1.261s user 0.000s sys 0.040s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785992227050689-9601-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785992227050689-9601-0/krb5kdc/test-user.keytab.
Aug 06 05:02:40 dist-test-slave-x4qh krb5kdc[8172](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992560, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T05:02:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T05:02:40Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785992227050689-9601-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785992227050689-9601-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 05:02:40.854900 9601 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
/tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785992227050689-9601-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785992227050689-9601-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785992227050689-9601-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785992227050689-9601-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36281
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785992227050689-9601-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43927
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36281
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785992227050689-9601-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785992227050689-9601-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785992227050689-9601-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 05:02:41.189344 8193 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 05:02:41.189640 8193 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 05:02:41.189844 8193 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 05:02:41.197072 8193 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 05:02:41.197175 8193 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 05:02:41.197295 8193 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 05:02:41.197397 8193 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 05:02:41.197475 8193 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 05:02:41.207159 8193 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43927
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785992227050689-9601-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785992227050689-9601-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36281
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785992227050689-9601-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36281
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785992227050689-9601-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785992227050689-9601-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 03:56:12 UTC on ccycloud.quasar-guijbm.root.comops.site
build id 4569618
ASAN enabled
I20260806 05:02:41.208580 8193 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 05:02:41.209869 8193 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 05:02:41.222152 8200 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 05:02:41.222092 8206 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 05:02:41.222124 8201 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 05:02:41.222774 8193 server_base.cc:1034] running on GCE node
I20260806 05:02:41.223745 8193 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 05:02:41.225167 8193 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 05:02:41.226432 8193 hybrid_clock.cc:648] HybridClock initialized: now 1785992561226376 us; error 49 us; skew 500 ppm
I20260806 05:02:41.226779 8193 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 05:02:41 dist-test-slave-x4qh krb5kdc[8172](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992561, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 05:02:41.235105 8193 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 05:02:41.236742 8193 webserver.cc:460] Webserver started at http://127.0.0.1:33553/ using document root <none> and password file <none>
I20260806 05:02:41.237461 8193 fs_manager.cc:359] Metadata directory not provided
I20260806 05:02:41.237619 8193 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 05:02:41.237943 8193 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 05:02:41.241045 8193 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785992227050689-9601-0/minicluster-data/master-0/data/instance:
uuid: "01d1e6f3c77e4011b57fa783f9c475d1"
format_stamp: "Formatted at 2026-08-06 05:02:41 on dist-test-slave-x4qh"
server_key: "c5508bd2e8337d3ed7da2c74ebf8cf3f"
server_key_iv: "9bc44aa1aa8c1cdcc71aa4620975c18c"
server_key_version: "encryptionkey@0"
I20260806 05:02:41.241847 8193 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785992227050689-9601-0/minicluster-data/master-0/wal/instance:
uuid: "01d1e6f3c77e4011b57fa783f9c475d1"
format_stamp: "Formatted at 2026-08-06 05:02:41 on dist-test-slave-x4qh"
server_key: "c5508bd2e8337d3ed7da2c74ebf8cf3f"
server_key_iv: "9bc44aa1aa8c1cdcc71aa4620975c18c"
server_key_version: "encryptionkey@0"
I20260806 05:02:41.247026 8193 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260806 05:02:41.250532 8213 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 05:02:41.251863 8193 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 05:02:41.252053 8193 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785992227050689-9601-0/minicluster-data/master-0/data,/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785992227050689-9601-0/minicluster-data/master-0/wal
uuid: "01d1e6f3c77e4011b57fa783f9c475d1"
format_stamp: "Formatted at 2026-08-06 05:02:41 on dist-test-slave-x4qh"
server_key: "c5508bd2e8337d3ed7da2c74ebf8cf3f"
server_key_iv: "9bc44aa1aa8c1cdcc71aa4620975c18c"
server_key_version: "encryptionkey@0"
I20260806 05:02:41.252271 8193 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785992227050689-9601-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785992227050689-9601-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785992227050689-9601-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 05:02:41.257982 8193 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 05:02:41.262310 8193 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 05:02:41.262622 8193 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 05:02:41.282163 8193 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36281
I20260806 05:02:41.282246 8302 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36281 every 8 connection(s)
I20260806 05:02:41.283658 8193 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785992227050689-9601-0/minicluster-data/master-0/data/info.pb
I20260806 05:02:41.287181 8303 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 05:02:41.293990 9601 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu as pid 8193
I20260806 05:02:41.294200 9601 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785992227050689-9601-0/minicluster-data/master-0/wal/instance
I20260806 05:02:41.294752 9601 external_mini_cluster.cc:1428] Setting key ef7aa1f8c2195714fdf0065ec1d2e515
I20260806 05:02:41.298274 8303 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 01d1e6f3c77e4011b57fa783f9c475d1: Bootstrap starting.
I20260806 05:02:41.301863 8303 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 01d1e6f3c77e4011b57fa783f9c475d1: Neither blocks nor log segments found. Creating new log.
I20260806 05:02:41.302625 8306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:02:41.296581 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43729 (local address 127.0.0.1:36281)
0806 05:02:41.297404 (+ 823us) server_negotiation.cc:206] Beginning negotiation
0806 05:02:41.297415 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 05:02:41.297457 (+ 42us) server_negotiation.cc:407] Connection header received
0806 05:02:41.299181 (+ 1724us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 05:02:41.299203 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 05:02:41.299644 (+ 441us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 05:02:41.299974 (+ 330us) server_negotiation.cc:226] Negotiated authn=SASL
0806 05:02:41.301062 (+ 1088us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 05:02:41.301068 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 05:02:41.301079 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 05:02:41.301093 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0806 05:02:41.301154 (+ 61us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 05:02:41.301172 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 05:02:41.301307 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0806 05:02:41.301479 (+ 172us) server_negotiation.cc:299] Negotiation successful
0806 05:02:41.301670 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":614,"thread_start_us":395,"threads_started":1}
I20260806 05:02:41.303076 8303 log.cc:824] T 00000000000000000000000000000000 P 01d1e6f3c77e4011b57fa783f9c475d1: Log is configured to *not* fsync() on all Append() calls
I20260806 05:02:41.306243 8303 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 01d1e6f3c77e4011b57fa783f9c475d1: No bootstrap required, opened a new log
I20260806 05:02:41.306469 9601 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 05:02:41.313525 8306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:02:41.311084 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55579 (local address 127.0.0.1:36281)
0806 05:02:41.311283 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0806 05:02:41.311290 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 05:02:41.311607 (+ 317us) server_negotiation.cc:407] Connection header received
0806 05:02:41.311794 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 05:02:41.311798 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 05:02:41.311848 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 05:02:41.311935 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0806 05:02:41.312850 (+ 915us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 05:02:41.312854 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 05:02:41.312857 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 05:02:41.312871 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0806 05:02:41.312911 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 05:02:41.312924 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 05:02:41.313025 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0806 05:02:41.313151 (+ 126us) server_negotiation.cc:299] Negotiation successful
0806 05:02:41.313195 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260806 05:02:41.314744 8303 raft_consensus.cc:348] T 00000000000000000000000000000000 P 01d1e6f3c77e4011b57fa783f9c475d1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01d1e6f3c77e4011b57fa783f9c475d1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36281 } }
I20260806 05:02:41.315114 8303 raft_consensus.cc:374] T 00000000000000000000000000000000 P 01d1e6f3c77e4011b57fa783f9c475d1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 05:02:41.315233 8303 raft_consensus.cc:729] T 00000000000000000000000000000000 P 01d1e6f3c77e4011b57fa783f9c475d1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 01d1e6f3c77e4011b57fa783f9c475d1, State: Initialized, Role: FOLLOWER
I20260806 05:02:41.315871 8303 consensus_queue.cc:260] T 00000000000000000000000000000000 P 01d1e6f3c77e4011b57fa783f9c475d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01d1e6f3c77e4011b57fa783f9c475d1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36281 } }
I20260806 05:02:41.316124 8303 raft_consensus.cc:388] T 00000000000000000000000000000000 P 01d1e6f3c77e4011b57fa783f9c475d1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 05:02:41.316282 8303 raft_consensus.cc:482] T 00000000000000000000000000000000 P 01d1e6f3c77e4011b57fa783f9c475d1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 05:02:41.316474 8303 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 01d1e6f3c77e4011b57fa783f9c475d1 [term 0 FOLLOWER]: Advancing to term 1
I20260806 05:02:41.318758 8303 raft_consensus.cc:504] T 00000000000000000000000000000000 P 01d1e6f3c77e4011b57fa783f9c475d1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01d1e6f3c77e4011b57fa783f9c475d1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36281 } }
I20260806 05:02:41.319252 8303 leader_election.cc:302] T 00000000000000000000000000000000 P 01d1e6f3c77e4011b57fa783f9c475d1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 01d1e6f3c77e4011b57fa783f9c475d1; no voters:
I20260806 05:02:41.319875 8303 leader_election.cc:288] T 00000000000000000000000000000000 P 01d1e6f3c77e4011b57fa783f9c475d1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 05:02:41.320008 8315 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 01d1e6f3c77e4011b57fa783f9c475d1 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 05:02:41.320785 8315 raft_consensus.cc:686] T 00000000000000000000000000000000 P 01d1e6f3c77e4011b57fa783f9c475d1 [term 1 LEADER]: Becoming Leader. State: Replica: 01d1e6f3c77e4011b57fa783f9c475d1, State: Running, Role: LEADER
I20260806 05:02:41.321333 8315 consensus_queue.cc:237] T 00000000000000000000000000000000 P 01d1e6f3c77e4011b57fa783f9c475d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01d1e6f3c77e4011b57fa783f9c475d1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36281 } }
I20260806 05:02:41.322002 8303 sys_catalog.cc:564] T 00000000000000000000000000000000 P 01d1e6f3c77e4011b57fa783f9c475d1 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 05:02:41.324882 8319 sys_catalog.cc:455] T 00000000000000000000000000000000 P 01d1e6f3c77e4011b57fa783f9c475d1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 01d1e6f3c77e4011b57fa783f9c475d1. Latest consensus state: current_term: 1 leader_uuid: "01d1e6f3c77e4011b57fa783f9c475d1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01d1e6f3c77e4011b57fa783f9c475d1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36281 } } }
I20260806 05:02:41.325374 8319 sys_catalog.cc:458] T 00000000000000000000000000000000 P 01d1e6f3c77e4011b57fa783f9c475d1 [sys.catalog]: This master's current role is: LEADER
I20260806 05:02:41.326004 8317 sys_catalog.cc:455] T 00000000000000000000000000000000 P 01d1e6f3c77e4011b57fa783f9c475d1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "01d1e6f3c77e4011b57fa783f9c475d1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01d1e6f3c77e4011b57fa783f9c475d1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36281 } } }
I20260806 05:02:41.326333 8317 sys_catalog.cc:458] T 00000000000000000000000000000000 P 01d1e6f3c77e4011b57fa783f9c475d1 [sys.catalog]: This master's current role is: LEADER
I20260806 05:02:41.327957 8326 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 05:02:41.335525 8326 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 05:02:41.346091 8326 catalog_manager.cc:1347] Generated new cluster ID: 98b1e4f85a6e40849541205dadf2269f
I20260806 05:02:41.346231 8326 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 05:02:41.365216 8326 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 05:02:41.365757 8326 catalog_manager.cc:1504] Loading token signing keys...
I20260806 05:02:41.374109 8326 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 01d1e6f3c77e4011b57fa783f9c475d1: Generated new TSK 0
I20260806 05:02:41.375064 8326 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 05:02:41.382318 9601 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu pid 8193
W20260806 05:02:41.383351 8295 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:43729
W20260806 05:02:41.384124 9601 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu pid 8193: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260806 05:02:41.384286 9601 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu with pid 8193
2026-08-06T05:02:41Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (2137 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785992227050689-9601-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 05:02:41 dist-test-slave-x4qh krb5kdc[8349](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 05:02:41 dist-test-slave-x4qh krb5kdc[8349](info): set up 2 sockets
Aug 06 05:02:41 dist-test-slave-x4qh krb5kdc[8349](info): commencing operation
krb5kdc: starting...
W20260806 05:02:42.705032 9601 mini_kdc.cc:121] Time spent starting KDC: real 1.275s user 0.000s sys 0.040s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785992227050689-9601-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785992227050689-9601-0/krb5kdc/test-user.keytab.
Aug 06 05:02:42 dist-test-slave-x4qh krb5kdc[8349](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992562, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T05:02:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T05:02:42Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785992227050689-9601-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785992227050689-9601-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 05:02:43.002491 9601 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
/tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785992227050689-9601-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785992227050689-9601-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785992227050689-9601-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785992227050689-9601-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36963
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785992227050689-9601-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38721
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36963
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785992227050689-9601-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785992227050689-9601-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785992227050689-9601-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 05:02:43.337888 8370 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 05:02:43.338210 8370 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 05:02:43.338379 8370 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 05:02:43.345455 8370 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 05:02:43.345609 8370 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 05:02:43.345715 8370 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 05:02:43.345831 8370 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 05:02:43.345909 8370 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 05:02:43.355576 8370 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38721
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785992227050689-9601-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785992227050689-9601-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36963
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785992227050689-9601-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36963
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785992227050689-9601-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785992227050689-9601-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 03:56:12 UTC on ccycloud.quasar-guijbm.root.comops.site
build id 4569618
ASAN enabled
I20260806 05:02:43.356814 8370 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 05:02:43.358090 8370 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 05:02:43.369880 8381 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 05:02:43.369930 8387 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 05:02:43.374033 8384 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 05:02:43.375591 8370 server_base.cc:1034] running on GCE node
I20260806 05:02:43.376405 8370 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 05:02:43.377708 8370 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 05:02:43.378943 8370 hybrid_clock.cc:648] HybridClock initialized: now 1785992563378881 us; error 53 us; skew 500 ppm
I20260806 05:02:43.379333 8370 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 05:02:43 dist-test-slave-x4qh krb5kdc[8349](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992563, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 05:02:43.382468 8370 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 05:02:43.383947 8370 webserver.cc:460] Webserver started at http://127.0.0.1:44883/ using document root <none> and password file <none>
I20260806 05:02:43.384610 8370 fs_manager.cc:359] Metadata directory not provided
I20260806 05:02:43.384722 8370 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 05:02:43.384974 8370 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 05:02:43.387880 8370 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785992227050689-9601-0/minicluster-data/master-0/data/instance:
uuid: "7328986b0a8e40d886845c2608cc4ffd"
format_stamp: "Formatted at 2026-08-06 05:02:43 on dist-test-slave-x4qh"
server_key: "09ae508be295ddc4a52f9fa52f6823d3"
server_key_iv: "69f6b3d3d8ebd404c85e95f56c6bd619"
server_key_version: "encryptionkey@0"
I20260806 05:02:43.388597 8370 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785992227050689-9601-0/minicluster-data/master-0/wal/instance:
uuid: "7328986b0a8e40d886845c2608cc4ffd"
format_stamp: "Formatted at 2026-08-06 05:02:43 on dist-test-slave-x4qh"
server_key: "09ae508be295ddc4a52f9fa52f6823d3"
server_key_iv: "69f6b3d3d8ebd404c85e95f56c6bd619"
server_key_version: "encryptionkey@0"
I20260806 05:02:43.393512 8370 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260806 05:02:43.396870 8398 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 05:02:43.398198 8370 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260806 05:02:43.398399 8370 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785992227050689-9601-0/minicluster-data/master-0/data,/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785992227050689-9601-0/minicluster-data/master-0/wal
uuid: "7328986b0a8e40d886845c2608cc4ffd"
format_stamp: "Formatted at 2026-08-06 05:02:43 on dist-test-slave-x4qh"
server_key: "09ae508be295ddc4a52f9fa52f6823d3"
server_key_iv: "69f6b3d3d8ebd404c85e95f56c6bd619"
server_key_version: "encryptionkey@0"
I20260806 05:02:43.398608 8370 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785992227050689-9601-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785992227050689-9601-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785992227050689-9601-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 05:02:43.418887 8370 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 05:02:43.422667 8370 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 05:02:43.422973 8370 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 05:02:43.441910 8370 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36963
I20260806 05:02:43.441934 8478 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36963 every 8 connection(s)
I20260806 05:02:43.443223 8370 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785992227050689-9601-0/minicluster-data/master-0/data/info.pb
I20260806 05:02:43.446923 8479 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 05:02:43.451982 9601 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu as pid 8370
I20260806 05:02:43.452172 9601 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785992227050689-9601-0/minicluster-data/master-0/wal/instance
I20260806 05:02:43.452639 9601 external_mini_cluster.cc:1428] Setting key 23847aa1c8bff7ee8f05b58f054209f9
I20260806 05:02:43.457883 8479 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7328986b0a8e40d886845c2608cc4ffd: Bootstrap starting.
I20260806 05:02:43.460865 8479 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7328986b0a8e40d886845c2608cc4ffd: Neither blocks nor log segments found. Creating new log.
I20260806 05:02:43.461839 8479 log.cc:824] T 00000000000000000000000000000000 P 7328986b0a8e40d886845c2608cc4ffd: Log is configured to *not* fsync() on all Append() calls
I20260806 05:02:43.464664 8479 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7328986b0a8e40d886845c2608cc4ffd: No bootstrap required, opened a new log
Aug 06 05:02:43 dist-test-slave-x4qh krb5kdc[8349](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992562, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260806 05:02:43.472532 8479 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7328986b0a8e40d886845c2608cc4ffd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7328986b0a8e40d886845c2608cc4ffd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36963 } }
I20260806 05:02:43.472892 8479 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7328986b0a8e40d886845c2608cc4ffd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 05:02:43.473042 8479 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7328986b0a8e40d886845c2608cc4ffd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7328986b0a8e40d886845c2608cc4ffd, State: Initialized, Role: FOLLOWER
I20260806 05:02:43.473690 8479 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7328986b0a8e40d886845c2608cc4ffd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7328986b0a8e40d886845c2608cc4ffd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36963 } }
I20260806 05:02:43.473946 8479 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7328986b0a8e40d886845c2608cc4ffd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 05:02:43.474100 8479 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7328986b0a8e40d886845c2608cc4ffd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 05:02:43.474298 8479 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7328986b0a8e40d886845c2608cc4ffd [term 0 FOLLOWER]: Advancing to term 1
I20260806 05:02:43.475010 8487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:02:43.454509 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47009 (local address 127.0.0.1:36963)
0806 05:02:43.455363 (+ 854us) server_negotiation.cc:206] Beginning negotiation
0806 05:02:43.455436 (+ 73us) server_negotiation.cc:399] Waiting for connection header
0806 05:02:43.455482 (+ 46us) server_negotiation.cc:407] Connection header received
0806 05:02:43.457188 (+ 1706us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 05:02:43.457211 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 05:02:43.457580 (+ 369us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 05:02:43.457974 (+ 394us) server_negotiation.cc:226] Negotiated authn=SASL
0806 05:02:43.459451 (+ 1477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:02:43.460739 (+ 1288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:02:43.462135 (+ 1396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:02:43.463237 (+ 1102us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:02:43.463473 (+ 236us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:02:43.467574 (+ 4101us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 05:02:43.467582 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 05:02:43.467595 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 05:02:43.467633 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0806 05:02:43.470799 (+ 3166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 05:02:43.471501 (+ 702us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 05:02:43.471516 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 05:02:43.471522 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 05:02:43.471694 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 05:02:43.472114 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 05:02:43.472119 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 05:02:43.472122 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 05:02:43.472633 (+ 511us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 05:02:43.472959 (+ 326us) server_negotiation.cc:1035] Waiting for connection context
0806 05:02:43.473395 (+ 436us) server_negotiation.cc:299] Negotiation successful
0806 05:02:43.473857 (+ 462us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":626,"thread_start_us":421,"threads_started":1}
I20260806 05:02:43.477169 8479 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7328986b0a8e40d886845c2608cc4ffd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7328986b0a8e40d886845c2608cc4ffd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36963 } }
I20260806 05:02:43.477699 8479 leader_election.cc:302] T 00000000000000000000000000000000 P 7328986b0a8e40d886845c2608cc4ffd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7328986b0a8e40d886845c2608cc4ffd; no voters:
I20260806 05:02:43.478416 8479 leader_election.cc:288] T 00000000000000000000000000000000 P 7328986b0a8e40d886845c2608cc4ffd [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 05:02:43.478542 8489 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7328986b0a8e40d886845c2608cc4ffd [term 1 FOLLOWER]: Leader election won for term 1
I20260806 05:02:43.478704 9601 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 05:02:43.480916 8479 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7328986b0a8e40d886845c2608cc4ffd [sys.catalog]: configured and running, proceeding with master startup.
I20260806 05:02:43.483124 8489 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7328986b0a8e40d886845c2608cc4ffd [term 1 LEADER]: Becoming Leader. State: Replica: 7328986b0a8e40d886845c2608cc4ffd, State: Running, Role: LEADER
I20260806 05:02:43.483934 8489 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7328986b0a8e40d886845c2608cc4ffd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7328986b0a8e40d886845c2608cc4ffd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36963 } }
I20260806 05:02:43.488185 8491 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7328986b0a8e40d886845c2608cc4ffd [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7328986b0a8e40d886845c2608cc4ffd. Latest consensus state: current_term: 1 leader_uuid: "7328986b0a8e40d886845c2608cc4ffd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7328986b0a8e40d886845c2608cc4ffd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36963 } } }
I20260806 05:02:43.488598 8491 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7328986b0a8e40d886845c2608cc4ffd [sys.catalog]: This master's current role is: LEADER
I20260806 05:02:43.489063 8490 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7328986b0a8e40d886845c2608cc4ffd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7328986b0a8e40d886845c2608cc4ffd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7328986b0a8e40d886845c2608cc4ffd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36963 } } }
I20260806 05:02:43.489347 8490 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7328986b0a8e40d886845c2608cc4ffd [sys.catalog]: This master's current role is: LEADER
I20260806 05:02:43.493353 8508 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 05:02:43.504021 8508 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 05:02:43.506064 8487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:02:43.483820 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37933 (local address 127.0.0.1:36963)
0806 05:02:43.483985 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0806 05:02:43.483990 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 05:02:43.484278 (+ 288us) server_negotiation.cc:407] Connection header received
0806 05:02:43.484523 (+ 245us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 05:02:43.484529 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 05:02:43.484601 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 05:02:43.484704 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0806 05:02:43.486732 (+ 2028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:02:43.487617 (+ 885us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:02:43.490008 (+ 2391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:02:43.490976 (+ 968us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:02:43.491115 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:02:43.494349 (+ 3234us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 05:02:43.494358 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 05:02:43.494363 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 05:02:43.494401 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0806 05:02:43.501922 (+ 7521us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 05:02:43.502798 (+ 876us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 05:02:43.502805 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 05:02:43.502810 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 05:02:43.502946 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 05:02:43.503465 (+ 519us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 05:02:43.503471 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 05:02:43.503475 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 05:02:43.503943 (+ 468us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 05:02:43.505229 (+ 1286us) server_negotiation.cc:1035] Waiting for connection context
0806 05:02:43.505401 (+ 172us) server_negotiation.cc:299] Negotiation successful
0806 05:02:43.505637 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260806 05:02:43.523180 8508 catalog_manager.cc:1347] Generated new cluster ID: 01932633dd01491595b6814a91ab7349
I20260806 05:02:43.523442 8508 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 05:02:43.546387 8508 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 05:02:43.547745 8508 catalog_manager.cc:1504] Loading token signing keys...
I20260806 05:02:43.555732 8508 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7328986b0a8e40d886845c2608cc4ffd: Generated new TSK 0
I20260806 05:02:43.556478 8508 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 05:02:43.566817 9601 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu pid 8370
I20260806 05:02:43.567826 8472 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 05:02:43.659899 9601 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu with pid 8370
2026-08-06T05:02:43Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (2272 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785992227050689-9601-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 05:02:43 dist-test-slave-x4qh krb5kdc[8532](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 05:02:43 dist-test-slave-x4qh krb5kdc[8532](info): set up 2 sockets
Aug 06 05:02:43 dist-test-slave-x4qh krb5kdc[8532](info): commencing operation
krb5kdc: starting...
W20260806 05:02:44.974785 9601 mini_kdc.cc:121] Time spent starting KDC: real 1.272s user 0.004s sys 0.033s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785992227050689-9601-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785992227050689-9601-0/krb5kdc/test-user.keytab.
Aug 06 05:02:45 dist-test-slave-x4qh krb5kdc[8532](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992565, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T05:02:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T05:02:45Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785992227050689-9601-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785992227050689-9601-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 05:02:45.259199 9601 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
/tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785992227050689-9601-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785992227050689-9601-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785992227050689-9601-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785992227050689-9601-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42607
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785992227050689-9601-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33675
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42607
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785992227050689-9601-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785992227050689-9601-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785992227050689-9601-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 05:02:45.594477 8551 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 05:02:45.594746 8551 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 05:02:45.594856 8551 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 05:02:45.601963 8551 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 05:02:45.602109 8551 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 05:02:45.602164 8551 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 05:02:45.602218 8551 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 05:02:45.602263 8551 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 05:02:45.611822 8551 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33675
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785992227050689-9601-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785992227050689-9601-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42607
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785992227050689-9601-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42607
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785992227050689-9601-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785992227050689-9601-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 03:56:12 UTC on ccycloud.quasar-guijbm.root.comops.site
build id 4569618
ASAN enabled
I20260806 05:02:45.612919 8551 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 05:02:45.614114 8551 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 05:02:45.625816 8558 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 05:02:45.625722 8562 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 05:02:45.625780 8559 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 05:02:45.626763 8551 server_base.cc:1034] running on GCE node
I20260806 05:02:45.627619 8551 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 05:02:45.628917 8551 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 05:02:45.630148 8551 hybrid_clock.cc:648] HybridClock initialized: now 1785992565630095 us; error 25 us; skew 500 ppm
I20260806 05:02:45.630503 8551 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 05:02:45 dist-test-slave-x4qh krb5kdc[8532](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992565, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 05:02:45.633621 8551 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 05:02:45.634974 8551 webserver.cc:460] Webserver started at http://127.0.0.1:41181/ using document root <none> and password file <none>
I20260806 05:02:45.635705 8551 fs_manager.cc:359] Metadata directory not provided
I20260806 05:02:45.635854 8551 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 05:02:45.636155 8551 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 05:02:45.638996 8551 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785992227050689-9601-0/minicluster-data/master-0/data/instance:
uuid: "4e9f6cb4da0b446da3744c3ddb9266a9"
format_stamp: "Formatted at 2026-08-06 05:02:45 on dist-test-slave-x4qh"
server_key: "1ef5628fced4f0fba224a6e698f3efcb"
server_key_iv: "0b4e9c04b27711fd6f3f2ee7e29fbe58"
server_key_version: "encryptionkey@0"
I20260806 05:02:45.639767 8551 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785992227050689-9601-0/minicluster-data/master-0/wal/instance:
uuid: "4e9f6cb4da0b446da3744c3ddb9266a9"
format_stamp: "Formatted at 2026-08-06 05:02:45 on dist-test-slave-x4qh"
server_key: "1ef5628fced4f0fba224a6e698f3efcb"
server_key_iv: "0b4e9c04b27711fd6f3f2ee7e29fbe58"
server_key_version: "encryptionkey@0"
I20260806 05:02:45.644760 8551 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260806 05:02:45.647970 8570 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 05:02:45.649121 8551 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.003s
I20260806 05:02:45.649314 8551 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785992227050689-9601-0/minicluster-data/master-0/data,/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785992227050689-9601-0/minicluster-data/master-0/wal
uuid: "4e9f6cb4da0b446da3744c3ddb9266a9"
format_stamp: "Formatted at 2026-08-06 05:02:45 on dist-test-slave-x4qh"
server_key: "1ef5628fced4f0fba224a6e698f3efcb"
server_key_iv: "0b4e9c04b27711fd6f3f2ee7e29fbe58"
server_key_version: "encryptionkey@0"
I20260806 05:02:45.649523 8551 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785992227050689-9601-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785992227050689-9601-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785992227050689-9601-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 05:02:45.668138 8551 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 05:02:45.672581 8551 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 05:02:45.672857 8551 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 05:02:45.691095 8551 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42607
I20260806 05:02:45.691174 8652 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42607 every 8 connection(s)
I20260806 05:02:45.692899 8551 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785992227050689-9601-0/minicluster-data/master-0/data/info.pb
I20260806 05:02:45.696346 9601 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu as pid 8551
I20260806 05:02:45.696534 9601 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785992227050689-9601-0/minicluster-data/master-0/wal/instance
I20260806 05:02:45.696911 8653 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 05:02:45.697094 9601 external_mini_cluster.cc:1428] Setting key 34df48a5e4fedad1880e8cccb2d9c5e1
I20260806 05:02:45.712133 8653 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4e9f6cb4da0b446da3744c3ddb9266a9: Bootstrap starting.
I20260806 05:02:45.712431 8657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:02:45.698963 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50205 (local address 127.0.0.1:42607)
0806 05:02:45.699846 (+ 883us) server_negotiation.cc:206] Beginning negotiation
0806 05:02:45.699872 (+ 26us) server_negotiation.cc:399] Waiting for connection header
0806 05:02:45.699927 (+ 55us) server_negotiation.cc:407] Connection header received
0806 05:02:45.703150 (+ 3223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 05:02:45.703178 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 05:02:45.703782 (+ 604us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 05:02:45.704281 (+ 499us) server_negotiation.cc:226] Negotiated authn=SASL
0806 05:02:45.704887 (+ 606us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:02:45.706274 (+ 1387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:02:45.708191 (+ 1917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:02:45.708877 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:02:45.709007 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:02:45.710205 (+ 1198us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 05:02:45.710216 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 05:02:45.710227 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 05:02:45.710726 (+ 499us) server_negotiation.cc:922] calling sasl_server_start()
0806 05:02:45.710798 (+ 72us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 05:02:45.710823 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 05:02:45.710924 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0806 05:02:45.711159 (+ 235us) server_negotiation.cc:299] Negotiation successful
0806 05:02:45.711470 (+ 311us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":632,"thread_start_us":392,"threads_started":1}
I20260806 05:02:45.715288 8653 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4e9f6cb4da0b446da3744c3ddb9266a9: Neither blocks nor log segments found. Creating new log.
I20260806 05:02:45.715971 9601 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 05:02:45.716470 8653 log.cc:824] T 00000000000000000000000000000000 P 4e9f6cb4da0b446da3744c3ddb9266a9: Log is configured to *not* fsync() on all Append() calls
I20260806 05:02:45.719667 8653 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4e9f6cb4da0b446da3744c3ddb9266a9: No bootstrap required, opened a new log
I20260806 05:02:45.727859 8657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:02:45.721381 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36893 (local address 127.0.0.1:42607)
0806 05:02:45.721579 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0806 05:02:45.721584 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 05:02:45.721960 (+ 376us) server_negotiation.cc:407] Connection header received
0806 05:02:45.722094 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 05:02:45.722101 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 05:02:45.722189 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 05:02:45.722298 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0806 05:02:45.722819 (+ 521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:02:45.723969 (+ 1150us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:02:45.725237 (+ 1268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:02:45.725948 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:02:45.726081 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:02:45.727064 (+ 983us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 05:02:45.727071 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 05:02:45.727076 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 05:02:45.727095 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0806 05:02:45.727154 (+ 59us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 05:02:45.727180 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 05:02:45.727295 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0806 05:02:45.727402 (+ 107us) server_negotiation.cc:299] Negotiation successful
0806 05:02:45.727507 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260806 05:02:45.729310 8653 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4e9f6cb4da0b446da3744c3ddb9266a9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e9f6cb4da0b446da3744c3ddb9266a9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42607 } }
I20260806 05:02:45.729630 8653 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4e9f6cb4da0b446da3744c3ddb9266a9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 05:02:45.729743 8653 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4e9f6cb4da0b446da3744c3ddb9266a9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4e9f6cb4da0b446da3744c3ddb9266a9, State: Initialized, Role: FOLLOWER
I20260806 05:02:45.730279 8653 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4e9f6cb4da0b446da3744c3ddb9266a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e9f6cb4da0b446da3744c3ddb9266a9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42607 } }
I20260806 05:02:45.730522 8653 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4e9f6cb4da0b446da3744c3ddb9266a9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 05:02:45.730624 8653 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4e9f6cb4da0b446da3744c3ddb9266a9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 05:02:45.730754 8653 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4e9f6cb4da0b446da3744c3ddb9266a9 [term 0 FOLLOWER]: Advancing to term 1
I20260806 05:02:45.733183 8653 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4e9f6cb4da0b446da3744c3ddb9266a9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e9f6cb4da0b446da3744c3ddb9266a9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42607 } }
I20260806 05:02:45.733649 8653 leader_election.cc:302] T 00000000000000000000000000000000 P 4e9f6cb4da0b446da3744c3ddb9266a9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4e9f6cb4da0b446da3744c3ddb9266a9; no voters:
I20260806 05:02:45.734227 8653 leader_election.cc:288] T 00000000000000000000000000000000 P 4e9f6cb4da0b446da3744c3ddb9266a9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 05:02:45.734356 8665 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4e9f6cb4da0b446da3744c3ddb9266a9 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 05:02:45.735651 8665 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4e9f6cb4da0b446da3744c3ddb9266a9 [term 1 LEADER]: Becoming Leader. State: Replica: 4e9f6cb4da0b446da3744c3ddb9266a9, State: Running, Role: LEADER
I20260806 05:02:45.736207 8665 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4e9f6cb4da0b446da3744c3ddb9266a9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e9f6cb4da0b446da3744c3ddb9266a9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42607 } }
I20260806 05:02:45.736696 8653 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4e9f6cb4da0b446da3744c3ddb9266a9 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 05:02:45.739184 8667 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4e9f6cb4da0b446da3744c3ddb9266a9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4e9f6cb4da0b446da3744c3ddb9266a9. Latest consensus state: current_term: 1 leader_uuid: "4e9f6cb4da0b446da3744c3ddb9266a9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e9f6cb4da0b446da3744c3ddb9266a9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42607 } } }
I20260806 05:02:45.739259 8666 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4e9f6cb4da0b446da3744c3ddb9266a9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4e9f6cb4da0b446da3744c3ddb9266a9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e9f6cb4da0b446da3744c3ddb9266a9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42607 } } }
I20260806 05:02:45.739607 8667 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4e9f6cb4da0b446da3744c3ddb9266a9 [sys.catalog]: This master's current role is: LEADER
I20260806 05:02:45.739693 8666 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4e9f6cb4da0b446da3744c3ddb9266a9 [sys.catalog]: This master's current role is: LEADER
I20260806 05:02:45.742249 8672 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 05:02:45.748426 8672 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 05:02:45.758373 8672 catalog_manager.cc:1347] Generated new cluster ID: 3f9cb3a8da9d4626a5195f53cf6e5c7b
I20260806 05:02:45.758493 8672 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 05:02:45.770186 8672 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 05:02:45.772070 8672 catalog_manager.cc:1504] Loading token signing keys...
I20260806 05:02:45.784659 8672 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4e9f6cb4da0b446da3744c3ddb9266a9: Generated new TSK 0
I20260806 05:02:45.785632 8672 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 05:02:45.791864 9601 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu pid 8551
W20260806 05:02:45.793071 8647 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:50205
W20260806 05:02:45.793706 9601 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu pid 8551: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260806 05:02:45.793861 9601 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu with pid 8551
2026-08-06T05:02:45Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (2137 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785992227050689-9601-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 05:02:45 dist-test-slave-x4qh krb5kdc[8692](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 05:02:45 dist-test-slave-x4qh krb5kdc[8692](info): set up 2 sockets
Aug 06 05:02:45 dist-test-slave-x4qh krb5kdc[8692](info): commencing operation
krb5kdc: starting...
W20260806 05:02:47.113027 9601 mini_kdc.cc:121] Time spent starting KDC: real 1.273s user 0.002s sys 0.037s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785992227050689-9601-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785992227050689-9601-0/krb5kdc/test-user.keytab.
Aug 06 05:02:47 dist-test-slave-x4qh krb5kdc[8692](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992567, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T05:02:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T05:02:47Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785992227050689-9601-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785992227050689-9601-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 05:02:47.426323 9601 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
/tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785992227050689-9601-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785992227050689-9601-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785992227050689-9601-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785992227050689-9601-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42235
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785992227050689-9601-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38491
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42235
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785992227050689-9601-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785992227050689-9601-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785992227050689-9601-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 05:02:47.779747 8711 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 05:02:47.780036 8711 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 05:02:47.780129 8711 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 05:02:47.787194 8711 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 05:02:47.787303 8711 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 05:02:47.787351 8711 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 05:02:47.787492 8711 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 05:02:47.787547 8711 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 05:02:47.797140 8711 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38491
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785992227050689-9601-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785992227050689-9601-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42235
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785992227050689-9601-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42235
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785992227050689-9601-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785992227050689-9601-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 03:56:12 UTC on ccycloud.quasar-guijbm.root.comops.site
build id 4569618
ASAN enabled
I20260806 05:02:47.798322 8711 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 05:02:47.799518 8711 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 05:02:47.811913 8718 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 05:02:47.812043 8720 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 05:02:47.812577 8717 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 05:02:47.812704 8711 server_base.cc:1034] running on GCE node
I20260806 05:02:47.813449 8711 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 05:02:47.814798 8711 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 05:02:47.816025 8711 hybrid_clock.cc:648] HybridClock initialized: now 1785992567815987 us; error 59 us; skew 500 ppm
I20260806 05:02:47.816362 8711 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 05:02:47 dist-test-slave-x4qh krb5kdc[8692](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992567, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 05:02:47.819469 8711 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 05:02:47.820940 8711 webserver.cc:460] Webserver started at http://127.0.0.1:42869/ using document root <none> and password file <none>
I20260806 05:02:47.821605 8711 fs_manager.cc:359] Metadata directory not provided
I20260806 05:02:47.821758 8711 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 05:02:47.822077 8711 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 05:02:47.825181 8711 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785992227050689-9601-0/minicluster-data/master-0/data/instance:
uuid: "56b835b715b948809b84fc43ce502ab3"
format_stamp: "Formatted at 2026-08-06 05:02:47 on dist-test-slave-x4qh"
server_key: "e8b45facf6e670707083cd5370f104e3"
server_key_iv: "42655b966fde7bb3aa131f0de4346963"
server_key_version: "encryptionkey@0"
I20260806 05:02:47.825958 8711 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785992227050689-9601-0/minicluster-data/master-0/wal/instance:
uuid: "56b835b715b948809b84fc43ce502ab3"
format_stamp: "Formatted at 2026-08-06 05:02:47 on dist-test-slave-x4qh"
server_key: "e8b45facf6e670707083cd5370f104e3"
server_key_iv: "42655b966fde7bb3aa131f0de4346963"
server_key_version: "encryptionkey@0"
I20260806 05:02:47.831578 8711 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260806 05:02:47.834903 8732 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 05:02:47.836179 8711 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 05:02:47.836367 8711 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785992227050689-9601-0/minicluster-data/master-0/data,/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785992227050689-9601-0/minicluster-data/master-0/wal
uuid: "56b835b715b948809b84fc43ce502ab3"
format_stamp: "Formatted at 2026-08-06 05:02:47 on dist-test-slave-x4qh"
server_key: "e8b45facf6e670707083cd5370f104e3"
server_key_iv: "42655b966fde7bb3aa131f0de4346963"
server_key_version: "encryptionkey@0"
I20260806 05:02:47.836571 8711 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785992227050689-9601-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785992227050689-9601-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785992227050689-9601-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 05:02:47.842129 8711 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 05:02:47.845969 8711 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 05:02:47.846275 8711 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 05:02:47.864848 8711 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42235
I20260806 05:02:47.864950 8806 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42235 every 8 connection(s)
I20260806 05:02:47.866220 8711 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785992227050689-9601-0/minicluster-data/master-0/data/info.pb
I20260806 05:02:47.869859 8807 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 05:02:47.875486 9601 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu as pid 8711
I20260806 05:02:47.875672 9601 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785992227050689-9601-0/minicluster-data/master-0/wal/instance
I20260806 05:02:47.876227 9601 external_mini_cluster.cc:1428] Setting key c29e7586dccc5a5a5aa9e7795adb2ec9
I20260806 05:02:47.881223 8807 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 56b835b715b948809b84fc43ce502ab3: Bootstrap starting.
I20260806 05:02:47.885375 8807 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 56b835b715b948809b84fc43ce502ab3: Neither blocks nor log segments found. Creating new log.
I20260806 05:02:47.886197 8813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:02:47.878295 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39847 (local address 127.0.0.1:42235)
0806 05:02:47.879291 (+ 996us) server_negotiation.cc:206] Beginning negotiation
0806 05:02:47.879307 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 05:02:47.879695 (+ 388us) server_negotiation.cc:407] Connection header received
0806 05:02:47.882111 (+ 2416us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 05:02:47.882137 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 05:02:47.882549 (+ 412us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 05:02:47.883022 (+ 473us) server_negotiation.cc:226] Negotiated authn=SASL
0806 05:02:47.884068 (+ 1046us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 05:02:47.884076 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 05:02:47.884092 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 05:02:47.884109 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0806 05:02:47.884182 (+ 73us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 05:02:47.884208 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 05:02:47.884446 (+ 238us) server_negotiation.cc:1035] Waiting for connection context
0806 05:02:47.884723 (+ 277us) server_negotiation.cc:299] Negotiation successful
0806 05:02:47.884908 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":726,"thread_start_us":445,"threads_started":1}
I20260806 05:02:47.886718 8807 log.cc:824] T 00000000000000000000000000000000 P 56b835b715b948809b84fc43ce502ab3: Log is configured to *not* fsync() on all Append() calls
I20260806 05:02:47.889853 8807 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 56b835b715b948809b84fc43ce502ab3: No bootstrap required, opened a new log
I20260806 05:02:47.890022 9601 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 05:02:47.897733 8813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:02:47.895265 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53011 (local address 127.0.0.1:42235)
0806 05:02:47.895531 (+ 266us) server_negotiation.cc:206] Beginning negotiation
0806 05:02:47.895537 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 05:02:47.895894 (+ 357us) server_negotiation.cc:407] Connection header received
0806 05:02:47.896022 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 05:02:47.896027 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 05:02:47.896087 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 05:02:47.896228 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0806 05:02:47.897081 (+ 853us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 05:02:47.897089 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 05:02:47.897098 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 05:02:47.897116 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0806 05:02:47.897167 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 05:02:47.897192 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 05:02:47.897334 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0806 05:02:47.897400 (+ 66us) server_negotiation.cc:299] Negotiation successful
0806 05:02:47.897453 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":118}
I20260806 05:02:47.900074 8807 raft_consensus.cc:348] T 00000000000000000000000000000000 P 56b835b715b948809b84fc43ce502ab3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56b835b715b948809b84fc43ce502ab3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42235 } }
I20260806 05:02:47.900422 8807 raft_consensus.cc:374] T 00000000000000000000000000000000 P 56b835b715b948809b84fc43ce502ab3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 05:02:47.900501 8807 raft_consensus.cc:729] T 00000000000000000000000000000000 P 56b835b715b948809b84fc43ce502ab3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 56b835b715b948809b84fc43ce502ab3, State: Initialized, Role: FOLLOWER
I20260806 05:02:47.900974 8807 consensus_queue.cc:260] T 00000000000000000000000000000000 P 56b835b715b948809b84fc43ce502ab3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56b835b715b948809b84fc43ce502ab3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42235 } }
I20260806 05:02:47.901197 8807 raft_consensus.cc:388] T 00000000000000000000000000000000 P 56b835b715b948809b84fc43ce502ab3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 05:02:47.901291 8807 raft_consensus.cc:482] T 00000000000000000000000000000000 P 56b835b715b948809b84fc43ce502ab3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 05:02:47.901474 8807 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 56b835b715b948809b84fc43ce502ab3 [term 0 FOLLOWER]: Advancing to term 1
I20260806 05:02:47.904079 8807 raft_consensus.cc:504] T 00000000000000000000000000000000 P 56b835b715b948809b84fc43ce502ab3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56b835b715b948809b84fc43ce502ab3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42235 } }
I20260806 05:02:47.904527 8807 leader_election.cc:302] T 00000000000000000000000000000000 P 56b835b715b948809b84fc43ce502ab3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 56b835b715b948809b84fc43ce502ab3; no voters:
I20260806 05:02:47.905261 8807 leader_election.cc:288] T 00000000000000000000000000000000 P 56b835b715b948809b84fc43ce502ab3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 05:02:47.905411 8823 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 56b835b715b948809b84fc43ce502ab3 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 05:02:47.906203 8823 raft_consensus.cc:686] T 00000000000000000000000000000000 P 56b835b715b948809b84fc43ce502ab3 [term 1 LEADER]: Becoming Leader. State: Replica: 56b835b715b948809b84fc43ce502ab3, State: Running, Role: LEADER
I20260806 05:02:47.906749 8823 consensus_queue.cc:237] T 00000000000000000000000000000000 P 56b835b715b948809b84fc43ce502ab3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56b835b715b948809b84fc43ce502ab3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42235 } }
I20260806 05:02:47.907483 8807 sys_catalog.cc:564] T 00000000000000000000000000000000 P 56b835b715b948809b84fc43ce502ab3 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 05:02:47.910140 8824 sys_catalog.cc:455] T 00000000000000000000000000000000 P 56b835b715b948809b84fc43ce502ab3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "56b835b715b948809b84fc43ce502ab3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56b835b715b948809b84fc43ce502ab3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42235 } } }
I20260806 05:02:47.910622 8824 sys_catalog.cc:458] T 00000000000000000000000000000000 P 56b835b715b948809b84fc43ce502ab3 [sys.catalog]: This master's current role is: LEADER
I20260806 05:02:47.911188 8827 sys_catalog.cc:455] T 00000000000000000000000000000000 P 56b835b715b948809b84fc43ce502ab3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 56b835b715b948809b84fc43ce502ab3. Latest consensus state: current_term: 1 leader_uuid: "56b835b715b948809b84fc43ce502ab3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56b835b715b948809b84fc43ce502ab3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42235 } } }
I20260806 05:02:47.911558 8827 sys_catalog.cc:458] T 00000000000000000000000000000000 P 56b835b715b948809b84fc43ce502ab3 [sys.catalog]: This master's current role is: LEADER
I20260806 05:02:47.912061 8835 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 05:02:47.917454 8835 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 05:02:47.926218 8835 catalog_manager.cc:1347] Generated new cluster ID: 464e76cd7b1e4a829968b9a6314e587e
I20260806 05:02:47.926317 8835 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 05:02:47.938144 8835 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 05:02:47.938536 8835 catalog_manager.cc:1504] Loading token signing keys...
I20260806 05:02:47.948269 8835 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 56b835b715b948809b84fc43ce502ab3: Generated new TSK 0
I20260806 05:02:47.949313 8835 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 05:02:47.964916 9601 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu pid 8711
W20260806 05:02:47.966035 8799 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:39847
W20260806 05:02:47.966706 9601 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu pid 8711: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260806 05:02:47.966881 9601 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu with pid 8711
2026-08-06T05:02:47Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (2173 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785992227050689-9601-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 05:02:48 dist-test-slave-x4qh krb5kdc[8854](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 05:02:48 dist-test-slave-x4qh krb5kdc[8854](info): set up 2 sockets
Aug 06 05:02:48 dist-test-slave-x4qh krb5kdc[8854](info): commencing operation
krb5kdc: starting...
W20260806 05:02:49.280030 9601 mini_kdc.cc:121] Time spent starting KDC: real 1.265s user 0.008s sys 0.032s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785992227050689-9601-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785992227050689-9601-0/krb5kdc/test-user.keytab.
Aug 06 05:02:49 dist-test-slave-x4qh krb5kdc[8854](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992569, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T05:02:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T05:02:49Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785992227050689-9601-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785992227050689-9601-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 05:02:49.588938 9601 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
/tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785992227050689-9601-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785992227050689-9601-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785992227050689-9601-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785992227050689-9601-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40669
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785992227050689-9601-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41723
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40669
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785992227050689-9601-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785992227050689-9601-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785992227050689-9601-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 05:02:49.922861 8872 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 05:02:49.923166 8872 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 05:02:49.923391 8872 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 05:02:49.930678 8872 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 05:02:49.930780 8872 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 05:02:49.930874 8872 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 05:02:49.930986 8872 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 05:02:49.931064 8872 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 05:02:49.940658 8872 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41723
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785992227050689-9601-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785992227050689-9601-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40669
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785992227050689-9601-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40669
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785992227050689-9601-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785992227050689-9601-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 03:56:12 UTC on ccycloud.quasar-guijbm.root.comops.site
build id 4569618
ASAN enabled
I20260806 05:02:49.941947 8872 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 05:02:49.943251 8872 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 05:02:49.955063 8879 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 05:02:49.955127 8878 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 05:02:49.955675 8872 server_base.cc:1034] running on GCE node
W20260806 05:02:49.955814 8882 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 05:02:49.956759 8872 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 05:02:49.958096 8872 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 05:02:49.959337 8872 hybrid_clock.cc:648] HybridClock initialized: now 1785992569959298 us; error 48 us; skew 500 ppm
I20260806 05:02:49.959736 8872 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 05:02:49 dist-test-slave-x4qh krb5kdc[8854](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992569, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 05:02:49.962798 8872 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 05:02:49.964305 8872 webserver.cc:460] Webserver started at http://127.0.0.1:46655/ using document root <none> and password file <none>
I20260806 05:02:49.965016 8872 fs_manager.cc:359] Metadata directory not provided
I20260806 05:02:49.965164 8872 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 05:02:49.965476 8872 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 05:02:49.968554 8872 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785992227050689-9601-0/minicluster-data/master-0/data/instance:
uuid: "6d912441293240bf9a6ef9d0067aaeb5"
format_stamp: "Formatted at 2026-08-06 05:02:49 on dist-test-slave-x4qh"
server_key: "687b4b9df1b7cb0efffaf4e4d14feb40"
server_key_iv: "94a2c89bf7d1d4b4ff2cedba227ab937"
server_key_version: "encryptionkey@0"
I20260806 05:02:49.969332 8872 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785992227050689-9601-0/minicluster-data/master-0/wal/instance:
uuid: "6d912441293240bf9a6ef9d0067aaeb5"
format_stamp: "Formatted at 2026-08-06 05:02:49 on dist-test-slave-x4qh"
server_key: "687b4b9df1b7cb0efffaf4e4d14feb40"
server_key_iv: "94a2c89bf7d1d4b4ff2cedba227ab937"
server_key_version: "encryptionkey@0"
I20260806 05:02:49.974512 8872 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260806 05:02:49.978060 8890 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 05:02:49.979333 8872 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260806 05:02:49.979576 8872 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785992227050689-9601-0/minicluster-data/master-0/data,/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785992227050689-9601-0/minicluster-data/master-0/wal
uuid: "6d912441293240bf9a6ef9d0067aaeb5"
format_stamp: "Formatted at 2026-08-06 05:02:49 on dist-test-slave-x4qh"
server_key: "687b4b9df1b7cb0efffaf4e4d14feb40"
server_key_iv: "94a2c89bf7d1d4b4ff2cedba227ab937"
server_key_version: "encryptionkey@0"
I20260806 05:02:49.979776 8872 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785992227050689-9601-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785992227050689-9601-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785992227050689-9601-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 05:02:50.004016 8872 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 05:02:50.008309 8872 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 05:02:50.008605 8872 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 05:02:50.027271 8966 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40669 every 8 connection(s)
I20260806 05:02:50.027267 8872 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40669
I20260806 05:02:50.028774 8872 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785992227050689-9601-0/minicluster-data/master-0/data/info.pb
I20260806 05:02:50.032441 8967 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 05:02:50.037817 9601 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu as pid 8872
I20260806 05:02:50.038020 9601 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785992227050689-9601-0/minicluster-data/master-0/wal/instance
I20260806 05:02:50.038473 9601 external_mini_cluster.cc:1428] Setting key 425161b7db9de124d5d0decefb65c16a
I20260806 05:02:50.044109 8967 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6d912441293240bf9a6ef9d0067aaeb5: Bootstrap starting.
I20260806 05:02:50.047808 8967 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6d912441293240bf9a6ef9d0067aaeb5: Neither blocks nor log segments found. Creating new log.
I20260806 05:02:50.048849 8967 log.cc:824] T 00000000000000000000000000000000 P 6d912441293240bf9a6ef9d0067aaeb5: Log is configured to *not* fsync() on all Append() calls
I20260806 05:02:50.051776 8967 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6d912441293240bf9a6ef9d0067aaeb5: No bootstrap required, opened a new log
Aug 06 05:02:50 dist-test-slave-x4qh krb5kdc[8854](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992569, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260806 05:02:50.061970 8967 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6d912441293240bf9a6ef9d0067aaeb5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d912441293240bf9a6ef9d0067aaeb5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40669 } }
I20260806 05:02:50.062445 8967 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6d912441293240bf9a6ef9d0067aaeb5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 05:02:50.062577 8967 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6d912441293240bf9a6ef9d0067aaeb5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6d912441293240bf9a6ef9d0067aaeb5, State: Initialized, Role: FOLLOWER
I20260806 05:02:50.063354 8970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:02:50.040364 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.93:59473 (local address 127.0.0.1:40669)
0806 05:02:50.041212 (+ 848us) server_negotiation.cc:206] Beginning negotiation
0806 05:02:50.041228 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 05:02:50.041287 (+ 59us) server_negotiation.cc:407] Connection header received
0806 05:02:50.043673 (+ 2386us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 05:02:50.043703 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 05:02:50.044180 (+ 477us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 05:02:50.044616 (+ 436us) server_negotiation.cc:226] Negotiated authn=SASL
0806 05:02:50.045899 (+ 1283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:02:50.047937 (+ 2038us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:02:50.049212 (+ 1275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:02:50.050050 (+ 838us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:02:50.050267 (+ 217us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:02:50.054400 (+ 4133us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 05:02:50.054411 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 05:02:50.054428 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 05:02:50.054476 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0806 05:02:50.058657 (+ 4181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 05:02:50.059485 (+ 828us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 05:02:50.059494 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 05:02:50.059503 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0806 05:02:50.059731 (+ 228us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 05:02:50.060157 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 05:02:50.060162 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 05:02:50.060166 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 05:02:50.060877 (+ 711us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 05:02:50.061174 (+ 297us) server_negotiation.cc:1035] Waiting for connection context
0806 05:02:50.061625 (+ 451us) server_negotiation.cc:299] Negotiation successful
0806 05:02:50.061957 (+ 332us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":593,"thread_start_us":351,"threads_started":1}
I20260806 05:02:50.063242 8967 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6d912441293240bf9a6ef9d0067aaeb5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d912441293240bf9a6ef9d0067aaeb5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40669 } }
I20260806 05:02:50.063591 8967 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6d912441293240bf9a6ef9d0067aaeb5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 05:02:50.063742 8967 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6d912441293240bf9a6ef9d0067aaeb5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 05:02:50.063956 8967 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6d912441293240bf9a6ef9d0067aaeb5 [term 0 FOLLOWER]: Advancing to term 1
I20260806 05:02:50.066905 8967 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6d912441293240bf9a6ef9d0067aaeb5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d912441293240bf9a6ef9d0067aaeb5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40669 } }
I20260806 05:02:50.067411 9601 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 05:02:50.067498 8967 leader_election.cc:302] T 00000000000000000000000000000000 P 6d912441293240bf9a6ef9d0067aaeb5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6d912441293240bf9a6ef9d0067aaeb5; no voters:
I20260806 05:02:50.068140 8967 leader_election.cc:288] T 00000000000000000000000000000000 P 6d912441293240bf9a6ef9d0067aaeb5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 05:02:50.068261 8972 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6d912441293240bf9a6ef9d0067aaeb5 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 05:02:50.070034 8967 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6d912441293240bf9a6ef9d0067aaeb5 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 05:02:50.076351 8972 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6d912441293240bf9a6ef9d0067aaeb5 [term 1 LEADER]: Becoming Leader. State: Replica: 6d912441293240bf9a6ef9d0067aaeb5, State: Running, Role: LEADER
I20260806 05:02:50.077148 8972 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6d912441293240bf9a6ef9d0067aaeb5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d912441293240bf9a6ef9d0067aaeb5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40669 } }
I20260806 05:02:50.081228 8973 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6d912441293240bf9a6ef9d0067aaeb5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6d912441293240bf9a6ef9d0067aaeb5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d912441293240bf9a6ef9d0067aaeb5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40669 } } }
I20260806 05:02:50.081625 8973 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6d912441293240bf9a6ef9d0067aaeb5 [sys.catalog]: This master's current role is: LEADER
I20260806 05:02:50.082123 8975 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6d912441293240bf9a6ef9d0067aaeb5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6d912441293240bf9a6ef9d0067aaeb5. Latest consensus state: current_term: 1 leader_uuid: "6d912441293240bf9a6ef9d0067aaeb5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d912441293240bf9a6ef9d0067aaeb5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40669 } } }
I20260806 05:02:50.082414 8975 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6d912441293240bf9a6ef9d0067aaeb5 [sys.catalog]: This master's current role is: LEADER
W20260806 05:02:50.089344 9001 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 6d912441293240bf9a6ef9d0067aaeb5: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 05:02:50.089493 9001 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 05:02:50.089733 9002 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 05:02:50.091666 9002 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 05:02:50.098070 8970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:02:50.079162 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.93:34457 (local address 127.0.0.1:40669)
0806 05:02:50.079356 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0806 05:02:50.079405 (+ 49us) server_negotiation.cc:399] Waiting for connection header
0806 05:02:50.081162 (+ 1757us) server_negotiation.cc:407] Connection header received
0806 05:02:50.081461 (+ 299us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 05:02:50.081467 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 05:02:50.081559 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 05:02:50.081642 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0806 05:02:50.084016 (+ 2374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:02:50.085380 (+ 1364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:02:50.086558 (+ 1178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:02:50.087226 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:02:50.087395 (+ 169us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:02:50.090691 (+ 3296us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 05:02:50.090698 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 05:02:50.090703 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 05:02:50.090740 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0806 05:02:50.094583 (+ 3843us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 05:02:50.095581 (+ 998us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 05:02:50.095587 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 05:02:50.095591 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 05:02:50.095727 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 05:02:50.096278 (+ 551us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 05:02:50.096284 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 05:02:50.096288 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 05:02:50.096692 (+ 404us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 05:02:50.096986 (+ 294us) server_negotiation.cc:1035] Waiting for connection context
0806 05:02:50.097428 (+ 442us) server_negotiation.cc:299] Negotiation successful
0806 05:02:50.097604 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260806 05:02:50.100065 9002 catalog_manager.cc:1347] Generated new cluster ID: b3ea006661714ef3b377e0c36af71e46
I20260806 05:02:50.100174 9002 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 05:02:50.109586 9002 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 05:02:50.110805 9002 catalog_manager.cc:1504] Loading token signing keys...
I20260806 05:02:50.117964 9002 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6d912441293240bf9a6ef9d0067aaeb5: Generated new TSK 0
I20260806 05:02:50.118827 9002 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 05:02:50.123775 9601 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu pid 8872
I20260806 05:02:50.124974 8962 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 05:02:50.213376 9601 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu with pid 8872
W20260806 05:02:50.237267 8851 connection.cc:535] client connection to 127.0.0.1:40669 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:40669): Connection reset by peer (error 104)
2026-08-06T05:02:50Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (2242 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785992227050689-9601-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 05:02:50 dist-test-slave-x4qh krb5kdc[9016](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 05:02:50 dist-test-slave-x4qh krb5kdc[9016](info): set up 2 sockets
Aug 06 05:02:50 dist-test-slave-x4qh krb5kdc[9016](info): commencing operation
krb5kdc: starting...
W20260806 05:02:51.531983 9601 mini_kdc.cc:121] Time spent starting KDC: real 1.275s user 0.007s sys 0.038s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785992227050689-9601-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785992227050689-9601-0/krb5kdc/test-user.keytab.
Aug 06 05:02:51 dist-test-slave-x4qh krb5kdc[9016](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992571, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T05:02:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T05:02:51Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785992227050689-9601-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785992227050689-9601-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 05:02:51.835003 9601 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
/tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785992227050689-9601-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785992227050689-9601-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785992227050689-9601-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785992227050689-9601-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43611
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785992227050689-9601-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45653
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43611
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785992227050689-9601-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785992227050689-9601-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785992227050689-9601-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 05:02:52.171310 9036 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 05:02:52.171605 9036 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 05:02:52.171823 9036 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 05:02:52.179143 9036 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 05:02:52.179250 9036 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 05:02:52.179354 9036 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 05:02:52.179486 9036 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 05:02:52.179590 9036 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 05:02:52.189287 9036 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45653
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785992227050689-9601-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785992227050689-9601-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43611
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785992227050689-9601-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43611
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785992227050689-9601-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785992227050689-9601-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 03:56:12 UTC on ccycloud.quasar-guijbm.root.comops.site
build id 4569618
ASAN enabled
I20260806 05:02:52.190546 9036 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 05:02:52.191887 9036 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 05:02:52.203986 9045 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 05:02:52.204092 9047 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 05:02:52.204119 9044 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 05:02:52.204730 9036 server_base.cc:1034] running on GCE node
I20260806 05:02:52.205560 9036 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 05:02:52.206880 9036 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 05:02:52.208115 9036 hybrid_clock.cc:648] HybridClock initialized: now 1785992572208085 us; error 50 us; skew 500 ppm
I20260806 05:02:52.208463 9036 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 05:02:52 dist-test-slave-x4qh krb5kdc[9016](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992572, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 05:02:52.211580 9036 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 05:02:52.213017 9036 webserver.cc:460] Webserver started at http://127.0.0.1:33941/ using document root <none> and password file <none>
I20260806 05:02:52.213688 9036 fs_manager.cc:359] Metadata directory not provided
I20260806 05:02:52.213845 9036 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 05:02:52.214170 9036 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 05:02:52.217206 9036 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785992227050689-9601-0/minicluster-data/master-0/data/instance:
uuid: "0ee823dbdad14b8894f728dbe80aaade"
format_stamp: "Formatted at 2026-08-06 05:02:52 on dist-test-slave-x4qh"
server_key: "0f561cf91c90c36cfc513b741ec86fe8"
server_key_iv: "d6395b8b3efb94de1fee9430de816f7f"
server_key_version: "encryptionkey@0"
I20260806 05:02:52.218005 9036 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785992227050689-9601-0/minicluster-data/master-0/wal/instance:
uuid: "0ee823dbdad14b8894f728dbe80aaade"
format_stamp: "Formatted at 2026-08-06 05:02:52 on dist-test-slave-x4qh"
server_key: "0f561cf91c90c36cfc513b741ec86fe8"
server_key_iv: "d6395b8b3efb94de1fee9430de816f7f"
server_key_version: "encryptionkey@0"
I20260806 05:02:52.223179 9036 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260806 05:02:52.226598 9056 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 05:02:52.227901 9036 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 05:02:52.228080 9036 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785992227050689-9601-0/minicluster-data/master-0/data,/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785992227050689-9601-0/minicluster-data/master-0/wal
uuid: "0ee823dbdad14b8894f728dbe80aaade"
format_stamp: "Formatted at 2026-08-06 05:02:52 on dist-test-slave-x4qh"
server_key: "0f561cf91c90c36cfc513b741ec86fe8"
server_key_iv: "d6395b8b3efb94de1fee9430de816f7f"
server_key_version: "encryptionkey@0"
I20260806 05:02:52.228278 9036 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785992227050689-9601-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785992227050689-9601-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785992227050689-9601-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 05:02:52.261238 9036 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 05:02:52.265909 9036 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 05:02:52.266234 9036 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 05:02:52.285392 9036 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43611
I20260806 05:02:52.285435 9137 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43611 every 8 connection(s)
I20260806 05:02:52.286764 9036 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785992227050689-9601-0/minicluster-data/master-0/data/info.pb
I20260806 05:02:52.290371 9139 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 05:02:52.294302 9601 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu as pid 9036
I20260806 05:02:52.294503 9601 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785992227050689-9601-0/minicluster-data/master-0/wal/instance
I20260806 05:02:52.294976 9601 external_mini_cluster.cc:1428] Setting key 257c36d336bae946d67b115e34e245c2
I20260806 05:02:52.303181 9139 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0ee823dbdad14b8894f728dbe80aaade: Bootstrap starting.
I20260806 05:02:52.306752 9139 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0ee823dbdad14b8894f728dbe80aaade: Neither blocks nor log segments found. Creating new log.
I20260806 05:02:52.308081 9139 log.cc:824] T 00000000000000000000000000000000 P 0ee823dbdad14b8894f728dbe80aaade: Log is configured to *not* fsync() on all Append() calls
I20260806 05:02:52.308421 9142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:02:52.296965 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.93:36305 (local address 127.0.0.1:43611)
0806 05:02:52.297816 (+ 851us) server_negotiation.cc:206] Beginning negotiation
0806 05:02:52.297828 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 05:02:52.297874 (+ 46us) server_negotiation.cc:407] Connection header received
0806 05:02:52.299478 (+ 1604us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 05:02:52.299505 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 05:02:52.299901 (+ 396us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 05:02:52.300206 (+ 305us) server_negotiation.cc:226] Negotiated authn=SASL
0806 05:02:52.300789 (+ 583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:02:52.302107 (+ 1318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:02:52.304045 (+ 1938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:02:52.304730 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:02:52.304855 (+ 125us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:02:52.306150 (+ 1295us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 05:02:52.306157 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 05:02:52.306170 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 05:02:52.306657 (+ 487us) server_negotiation.cc:922] calling sasl_server_start()
0806 05:02:52.306727 (+ 70us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 05:02:52.306753 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 05:02:52.306958 (+ 205us) server_negotiation.cc:1035] Waiting for connection context
0806 05:02:52.307217 (+ 259us) server_negotiation.cc:299] Negotiation successful
0806 05:02:52.307439 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":629,"thread_start_us":400,"threads_started":1}
I20260806 05:02:52.310770 9139 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0ee823dbdad14b8894f728dbe80aaade: No bootstrap required, opened a new log
I20260806 05:02:52.312110 9601 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 05:02:52.321448 9139 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0ee823dbdad14b8894f728dbe80aaade [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0ee823dbdad14b8894f728dbe80aaade" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43611 } }
I20260806 05:02:52.321957 9139 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0ee823dbdad14b8894f728dbe80aaade [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 05:02:52.322093 9139 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0ee823dbdad14b8894f728dbe80aaade [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ee823dbdad14b8894f728dbe80aaade, State: Initialized, Role: FOLLOWER
I20260806 05:02:52.322811 9139 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0ee823dbdad14b8894f728dbe80aaade [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0ee823dbdad14b8894f728dbe80aaade" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43611 } }
I20260806 05:02:52.323124 9139 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0ee823dbdad14b8894f728dbe80aaade [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 05:02:52.323267 9139 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0ee823dbdad14b8894f728dbe80aaade [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 05:02:52.323472 9139 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0ee823dbdad14b8894f728dbe80aaade [term 0 FOLLOWER]: Advancing to term 1
I20260806 05:02:52.324455 9142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:02:52.317270 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.93:53373 (local address 127.0.0.1:43611)
0806 05:02:52.317514 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0806 05:02:52.317522 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 05:02:52.317925 (+ 403us) server_negotiation.cc:407] Connection header received
0806 05:02:52.318096 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 05:02:52.318100 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 05:02:52.318157 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 05:02:52.318239 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0806 05:02:52.318828 (+ 589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:02:52.319883 (+ 1055us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:02:52.321714 (+ 1831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:02:52.322292 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:02:52.322414 (+ 122us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:02:52.323677 (+ 1263us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 05:02:52.323683 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 05:02:52.323689 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 05:02:52.323712 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0806 05:02:52.323773 (+ 61us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 05:02:52.323788 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 05:02:52.323918 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0806 05:02:52.324118 (+ 200us) server_negotiation.cc:299] Negotiation successful
0806 05:02:52.324178 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260806 05:02:52.326205 9139 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0ee823dbdad14b8894f728dbe80aaade [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0ee823dbdad14b8894f728dbe80aaade" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43611 } }
I20260806 05:02:52.326683 9139 leader_election.cc:302] T 00000000000000000000000000000000 P 0ee823dbdad14b8894f728dbe80aaade [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0ee823dbdad14b8894f728dbe80aaade; no voters:
I20260806 05:02:52.327241 9139 leader_election.cc:288] T 00000000000000000000000000000000 P 0ee823dbdad14b8894f728dbe80aaade [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 05:02:52.327418 9151 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0ee823dbdad14b8894f728dbe80aaade [term 1 FOLLOWER]: Leader election won for term 1
I20260806 05:02:52.328111 9151 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0ee823dbdad14b8894f728dbe80aaade [term 1 LEADER]: Becoming Leader. State: Replica: 0ee823dbdad14b8894f728dbe80aaade, State: Running, Role: LEADER
I20260806 05:02:52.328609 9151 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0ee823dbdad14b8894f728dbe80aaade [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0ee823dbdad14b8894f728dbe80aaade" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43611 } }
I20260806 05:02:52.329326 9139 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0ee823dbdad14b8894f728dbe80aaade [sys.catalog]: configured and running, proceeding with master startup.
I20260806 05:02:52.331562 9154 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0ee823dbdad14b8894f728dbe80aaade [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0ee823dbdad14b8894f728dbe80aaade. Latest consensus state: current_term: 1 leader_uuid: "0ee823dbdad14b8894f728dbe80aaade" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0ee823dbdad14b8894f728dbe80aaade" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43611 } } }
I20260806 05:02:52.332056 9154 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0ee823dbdad14b8894f728dbe80aaade [sys.catalog]: This master's current role is: LEADER
I20260806 05:02:52.332813 9153 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0ee823dbdad14b8894f728dbe80aaade [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0ee823dbdad14b8894f728dbe80aaade" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0ee823dbdad14b8894f728dbe80aaade" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43611 } } }
I20260806 05:02:52.333139 9153 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0ee823dbdad14b8894f728dbe80aaade [sys.catalog]: This master's current role is: LEADER
I20260806 05:02:52.336807 9164 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 05:02:52.341435 9164 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 05:02:52.349932 9164 catalog_manager.cc:1347] Generated new cluster ID: 11dfe8e7c0bc48698cd7b7b8228bd31b
I20260806 05:02:52.350034 9164 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 05:02:52.390422 9164 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 05:02:52.391917 9164 catalog_manager.cc:1504] Loading token signing keys...
I20260806 05:02:52.412137 9164 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0ee823dbdad14b8894f728dbe80aaade: Generated new TSK 0
I20260806 05:02:52.413043 9164 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 05:02:52.425851 9601 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu pid 9036
W20260806 05:02:52.427202 9133 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.93:36305
W20260806 05:02:52.427928 9601 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu pid 9036: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260806 05:02:52.428083 9601 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu with pid 9036
W20260806 05:02:52.454502 9014 connection.cc:535] client connection to 127.0.0.1:43611 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:43611): Connection reset by peer (error 104)
2026-08-06T05:02:52Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (2217 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785992227050689-9601-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 05:02:52 dist-test-slave-x4qh krb5kdc[9178](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 05:02:52 dist-test-slave-x4qh krb5kdc[9178](info): set up 2 sockets
Aug 06 05:02:52 dist-test-slave-x4qh krb5kdc[9178](info): commencing operation
krb5kdc: starting...
W20260806 05:02:53.762332 9601 mini_kdc.cc:121] Time spent starting KDC: real 1.288s user 0.000s sys 0.042s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785992227050689-9601-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785992227050689-9601-0/krb5kdc/test-user.keytab.
Aug 06 05:02:53 dist-test-slave-x4qh krb5kdc[9178](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992573, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T05:02:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T05:02:53Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785992227050689-9601-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785992227050689-9601-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 05:02:54.078588 9601 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
/tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785992227050689-9601-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785992227050689-9601-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785992227050689-9601-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785992227050689-9601-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45421
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785992227050689-9601-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33617
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:45421
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785992227050689-9601-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785992227050689-9601-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785992227050689-9601-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 05:02:54.416675 9201 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 05:02:54.416975 9201 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 05:02:54.417152 9201 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 05:02:54.424283 9201 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 05:02:54.424439 9201 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 05:02:54.424556 9201 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 05:02:54.424662 9201 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 05:02:54.424753 9201 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 05:02:54.434473 9201 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33617
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785992227050689-9601-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785992227050689-9601-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:45421
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785992227050689-9601-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45421
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785992227050689-9601-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785992227050689-9601-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 03:56:12 UTC on ccycloud.quasar-guijbm.root.comops.site
build id 4569618
ASAN enabled
I20260806 05:02:54.435894 9201 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 05:02:54.437204 9201 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 05:02:54.449134 9211 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 05:02:54.449205 9206 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 05:02:54.449201 9207 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 05:02:54.450002 9201 server_base.cc:1034] running on GCE node
I20260806 05:02:54.450785 9201 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 05:02:54.452240 9201 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 05:02:54.453449 9201 hybrid_clock.cc:648] HybridClock initialized: now 1785992574453402 us; error 34 us; skew 500 ppm
I20260806 05:02:54.453831 9201 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 05:02:54 dist-test-slave-x4qh krb5kdc[9178](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992574, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 05:02:54.461625 9201 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 05:02:54.463250 9201 webserver.cc:460] Webserver started at http://127.0.0.1:45251/ using document root <none> and password file <none>
I20260806 05:02:54.464035 9201 fs_manager.cc:359] Metadata directory not provided
I20260806 05:02:54.464171 9201 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 05:02:54.464533 9201 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 05:02:54.467695 9201 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785992227050689-9601-0/minicluster-data/master-0/data/instance:
uuid: "216fec085ebb4a5ea8a6bc7d8487c061"
format_stamp: "Formatted at 2026-08-06 05:02:54 on dist-test-slave-x4qh"
server_key: "c1fd93130b9951d767eaf64b9073b2ed"
server_key_iv: "46fd80072f41607358f8be6dcf237de6"
server_key_version: "encryptionkey@0"
I20260806 05:02:54.468488 9201 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785992227050689-9601-0/minicluster-data/master-0/wal/instance:
uuid: "216fec085ebb4a5ea8a6bc7d8487c061"
format_stamp: "Formatted at 2026-08-06 05:02:54 on dist-test-slave-x4qh"
server_key: "c1fd93130b9951d767eaf64b9073b2ed"
server_key_iv: "46fd80072f41607358f8be6dcf237de6"
server_key_version: "encryptionkey@0"
I20260806 05:02:54.473737 9201 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.003s
I20260806 05:02:54.477272 9220 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 05:02:54.478549 9201 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 05:02:54.478732 9201 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785992227050689-9601-0/minicluster-data/master-0/data,/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785992227050689-9601-0/minicluster-data/master-0/wal
uuid: "216fec085ebb4a5ea8a6bc7d8487c061"
format_stamp: "Formatted at 2026-08-06 05:02:54 on dist-test-slave-x4qh"
server_key: "c1fd93130b9951d767eaf64b9073b2ed"
server_key_iv: "46fd80072f41607358f8be6dcf237de6"
server_key_version: "encryptionkey@0"
I20260806 05:02:54.478958 9201 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785992227050689-9601-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785992227050689-9601-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785992227050689-9601-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 05:02:54.484632 9201 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 05:02:54.488468 9201 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 05:02:54.488761 9201 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 05:02:54.507893 9201 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:45421
I20260806 05:02:54.507973 9298 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:45421 every 8 connection(s)
I20260806 05:02:54.509248 9201 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785992227050689-9601-0/minicluster-data/master-0/data/info.pb
I20260806 05:02:54.512806 9299 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 05:02:54.518931 9601 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu as pid 9201
I20260806 05:02:54.519124 9601 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785992227050689-9601-0/minicluster-data/master-0/wal/instance
I20260806 05:02:54.519735 9601 external_mini_cluster.cc:1428] Setting key ebd7b93921b37bfd4dc0dc61ba5998c7
I20260806 05:02:54.523849 9299 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 216fec085ebb4a5ea8a6bc7d8487c061: Bootstrap starting.
I20260806 05:02:54.527035 9299 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 216fec085ebb4a5ea8a6bc7d8487c061: Neither blocks nor log segments found. Creating new log.
I20260806 05:02:54.527650 9303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:02:54.521633 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.93:55837 (local address 127.0.0.1:45421)
0806 05:02:54.522433 (+ 800us) server_negotiation.cc:206] Beginning negotiation
0806 05:02:54.522445 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 05:02:54.522484 (+ 39us) server_negotiation.cc:407] Connection header received
0806 05:02:54.524317 (+ 1833us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 05:02:54.524343 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 05:02:54.524657 (+ 314us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 05:02:54.525038 (+ 381us) server_negotiation.cc:226] Negotiated authn=SASL
0806 05:02:54.526061 (+ 1023us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 05:02:54.526067 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 05:02:54.526079 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 05:02:54.526092 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0806 05:02:54.526149 (+ 57us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 05:02:54.526170 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 05:02:54.526387 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0806 05:02:54.526528 (+ 141us) server_negotiation.cc:299] Negotiation successful
0806 05:02:54.526700 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":578,"thread_start_us":350,"threads_started":1}
I20260806 05:02:54.528191 9299 log.cc:824] T 00000000000000000000000000000000 P 216fec085ebb4a5ea8a6bc7d8487c061: Log is configured to *not* fsync() on all Append() calls
I20260806 05:02:54.531013 9299 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 216fec085ebb4a5ea8a6bc7d8487c061: No bootstrap required, opened a new log
I20260806 05:02:54.531183 9601 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 05:02:54.538458 9299 raft_consensus.cc:348] T 00000000000000000000000000000000 P 216fec085ebb4a5ea8a6bc7d8487c061 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "216fec085ebb4a5ea8a6bc7d8487c061" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45421 } }
I20260806 05:02:54.538812 9299 raft_consensus.cc:374] T 00000000000000000000000000000000 P 216fec085ebb4a5ea8a6bc7d8487c061 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 05:02:54.538931 9299 raft_consensus.cc:729] T 00000000000000000000000000000000 P 216fec085ebb4a5ea8a6bc7d8487c061 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 216fec085ebb4a5ea8a6bc7d8487c061, State: Initialized, Role: FOLLOWER
I20260806 05:02:54.539662 9299 consensus_queue.cc:260] T 00000000000000000000000000000000 P 216fec085ebb4a5ea8a6bc7d8487c061 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "216fec085ebb4a5ea8a6bc7d8487c061" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45421 } }
I20260806 05:02:54.539975 9299 raft_consensus.cc:388] T 00000000000000000000000000000000 P 216fec085ebb4a5ea8a6bc7d8487c061 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 05:02:54.540148 9299 raft_consensus.cc:482] T 00000000000000000000000000000000 P 216fec085ebb4a5ea8a6bc7d8487c061 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 05:02:54.540343 9299 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 216fec085ebb4a5ea8a6bc7d8487c061 [term 0 FOLLOWER]: Advancing to term 1
I20260806 05:02:54.541457 9303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:02:54.535668 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.93:55615 (local address 127.0.0.1:45421)
0806 05:02:54.535894 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0806 05:02:54.535900 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 05:02:54.539311 (+ 3411us) server_negotiation.cc:407] Connection header received
0806 05:02:54.539511 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 05:02:54.539515 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 05:02:54.539559 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 05:02:54.539639 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0806 05:02:54.540763 (+ 1124us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 05:02:54.540769 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 05:02:54.540773 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 05:02:54.540796 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0806 05:02:54.540834 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 05:02:54.540849 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 05:02:54.540957 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0806 05:02:54.541088 (+ 131us) server_negotiation.cc:299] Negotiation successful
0806 05:02:54.541141 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260806 05:02:54.543017 9299 raft_consensus.cc:504] T 00000000000000000000000000000000 P 216fec085ebb4a5ea8a6bc7d8487c061 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "216fec085ebb4a5ea8a6bc7d8487c061" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45421 } }
I20260806 05:02:54.543581 9299 leader_election.cc:302] T 00000000000000000000000000000000 P 216fec085ebb4a5ea8a6bc7d8487c061 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 216fec085ebb4a5ea8a6bc7d8487c061; no voters:
I20260806 05:02:54.544126 9299 leader_election.cc:288] T 00000000000000000000000000000000 P 216fec085ebb4a5ea8a6bc7d8487c061 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 05:02:54.544261 9315 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 216fec085ebb4a5ea8a6bc7d8487c061 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 05:02:54.545017 9315 raft_consensus.cc:686] T 00000000000000000000000000000000 P 216fec085ebb4a5ea8a6bc7d8487c061 [term 1 LEADER]: Becoming Leader. State: Replica: 216fec085ebb4a5ea8a6bc7d8487c061, State: Running, Role: LEADER
I20260806 05:02:54.545531 9315 consensus_queue.cc:237] T 00000000000000000000000000000000 P 216fec085ebb4a5ea8a6bc7d8487c061 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "216fec085ebb4a5ea8a6bc7d8487c061" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45421 } }
I20260806 05:02:54.546267 9299 sys_catalog.cc:564] T 00000000000000000000000000000000 P 216fec085ebb4a5ea8a6bc7d8487c061 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 05:02:54.548525 9317 sys_catalog.cc:455] T 00000000000000000000000000000000 P 216fec085ebb4a5ea8a6bc7d8487c061 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 216fec085ebb4a5ea8a6bc7d8487c061. Latest consensus state: current_term: 1 leader_uuid: "216fec085ebb4a5ea8a6bc7d8487c061" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "216fec085ebb4a5ea8a6bc7d8487c061" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45421 } } }
I20260806 05:02:54.548514 9316 sys_catalog.cc:455] T 00000000000000000000000000000000 P 216fec085ebb4a5ea8a6bc7d8487c061 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "216fec085ebb4a5ea8a6bc7d8487c061" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "216fec085ebb4a5ea8a6bc7d8487c061" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45421 } } }
I20260806 05:02:54.548979 9317 sys_catalog.cc:458] T 00000000000000000000000000000000 P 216fec085ebb4a5ea8a6bc7d8487c061 [sys.catalog]: This master's current role is: LEADER
I20260806 05:02:54.548979 9316 sys_catalog.cc:458] T 00000000000000000000000000000000 P 216fec085ebb4a5ea8a6bc7d8487c061 [sys.catalog]: This master's current role is: LEADER
I20260806 05:02:54.552080 9320 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 05:02:54.557521 9320 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 05:02:54.566952 9320 catalog_manager.cc:1347] Generated new cluster ID: 9b7cebae97f143eea8074ae22c1a766a
I20260806 05:02:54.567070 9320 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 05:02:54.593569 9320 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 05:02:54.594066 9320 catalog_manager.cc:1504] Loading token signing keys...
I20260806 05:02:54.604343 9320 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 216fec085ebb4a5ea8a6bc7d8487c061: Generated new TSK 0
I20260806 05:02:54.605381 9320 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 05:02:54.612146 9601 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu pid 9201
W20260806 05:02:54.613305 9291 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.93:55837
W20260806 05:02:54.614054 9601 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu pid 9201: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260806 05:02:54.614212 9601 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu with pid 9201
2026-08-06T05:02:54Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (2186 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785992227050689-9601-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 05:02:54 dist-test-slave-x4qh krb5kdc[9342](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 05:02:54 dist-test-slave-x4qh krb5kdc[9342](info): set up 2 sockets
Aug 06 05:02:54 dist-test-slave-x4qh krb5kdc[9342](info): commencing operation
krb5kdc: starting...
W20260806 05:02:55.974112 9601 mini_kdc.cc:121] Time spent starting KDC: real 1.313s user 0.000s sys 0.041s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785992227050689-9601-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785992227050689-9601-0/krb5kdc/test-user.keytab.
Aug 06 05:02:56 dist-test-slave-x4qh krb5kdc[9342](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992576, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T05:02:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T05:02:56Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785992227050689-9601-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785992227050689-9601-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 05:02:56.276759 9601 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
/tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785992227050689-9601-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785992227050689-9601-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785992227050689-9601-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785992227050689-9601-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44765
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785992227050689-9601-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37671
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44765
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785992227050689-9601-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785992227050689-9601-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785992227050689-9601-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 05:02:56.614045 9358 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 05:02:56.614315 9358 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 05:02:56.614406 9358 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 05:02:56.621346 9358 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 05:02:56.621448 9358 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 05:02:56.621495 9358 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 05:02:56.621557 9358 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 05:02:56.621601 9358 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 05:02:56.630874 9358 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37671
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785992227050689-9601-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785992227050689-9601-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44765
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785992227050689-9601-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44765
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785992227050689-9601-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785992227050689-9601-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 03:56:12 UTC on ccycloud.quasar-guijbm.root.comops.site
build id 4569618
ASAN enabled
I20260806 05:02:56.632042 9358 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 05:02:56.633270 9358 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 05:02:56.644759 9369 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 05:02:56.644836 9372 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 05:02:56.644856 9367 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 05:02:56.645754 9358 server_base.cc:1034] running on GCE node
I20260806 05:02:56.646517 9358 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 05:02:56.647821 9358 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 05:02:56.649063 9358 hybrid_clock.cc:648] HybridClock initialized: now 1785992576649036 us; error 50 us; skew 500 ppm
I20260806 05:02:56.649403 9358 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 05:02:56 dist-test-slave-x4qh krb5kdc[9342](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992576, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 05:02:56.652400 9358 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 05:02:56.653753 9358 webserver.cc:460] Webserver started at http://127.0.0.1:39315/ using document root <none> and password file <none>
I20260806 05:02:56.654421 9358 fs_manager.cc:359] Metadata directory not provided
I20260806 05:02:56.654588 9358 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 05:02:56.654903 9358 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 05:02:56.657797 9358 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785992227050689-9601-0/minicluster-data/master-0/data/instance:
uuid: "896454a20ba244ab83d5d0181f4f305e"
format_stamp: "Formatted at 2026-08-06 05:02:56 on dist-test-slave-x4qh"
server_key: "8ded9fb4767436eda2e806db7fa213b6"
server_key_iv: "cddd33c858f4e11ae661c5dc82306a74"
server_key_version: "encryptionkey@0"
I20260806 05:02:56.658597 9358 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785992227050689-9601-0/minicluster-data/master-0/wal/instance:
uuid: "896454a20ba244ab83d5d0181f4f305e"
format_stamp: "Formatted at 2026-08-06 05:02:56 on dist-test-slave-x4qh"
server_key: "8ded9fb4767436eda2e806db7fa213b6"
server_key_iv: "cddd33c858f4e11ae661c5dc82306a74"
server_key_version: "encryptionkey@0"
I20260806 05:02:56.663586 9358 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.008s sys 0.000s
I20260806 05:02:56.666859 9388 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 05:02:56.668308 9358 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 05:02:56.668498 9358 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785992227050689-9601-0/minicluster-data/master-0/data,/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785992227050689-9601-0/minicluster-data/master-0/wal
uuid: "896454a20ba244ab83d5d0181f4f305e"
format_stamp: "Formatted at 2026-08-06 05:02:56 on dist-test-slave-x4qh"
server_key: "8ded9fb4767436eda2e806db7fa213b6"
server_key_iv: "cddd33c858f4e11ae661c5dc82306a74"
server_key_version: "encryptionkey@0"
I20260806 05:02:56.668697 9358 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785992227050689-9601-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785992227050689-9601-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785992227050689-9601-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 05:02:56.685046 9358 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 05:02:56.689014 9358 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 05:02:56.689297 9358 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 05:02:56.708647 9358 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44765
I20260806 05:02:56.708680 9473 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44765 every 8 connection(s)
I20260806 05:02:56.709976 9358 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785992227050689-9601-0/minicluster-data/master-0/data/info.pb
I20260806 05:02:56.713603 9474 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 05:02:56.716320 9601 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu as pid 9358
I20260806 05:02:56.716526 9601 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785992227050689-9601-0/minicluster-data/master-0/wal/instance
I20260806 05:02:56.717073 9601 external_mini_cluster.cc:1428] Setting key a7c7b59e5c5e1cc788c22cf15588399c
I20260806 05:02:56.726272 9474 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 896454a20ba244ab83d5d0181f4f305e: Bootstrap starting.
I20260806 05:02:56.730196 9474 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 896454a20ba244ab83d5d0181f4f305e: Neither blocks nor log segments found. Creating new log.
I20260806 05:02:56.731609 9474 log.cc:824] T 00000000000000000000000000000000 P 896454a20ba244ab83d5d0181f4f305e: Log is configured to *not* fsync() on all Append() calls
Aug 06 05:02:56 dist-test-slave-x4qh krb5kdc[9342](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992576, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260806 05:02:56.734570 9474 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 896454a20ba244ab83d5d0181f4f305e: No bootstrap required, opened a new log
I20260806 05:02:56.741088 9479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:02:56.718942 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.93:37587 (local address 127.0.0.1:44765)
0806 05:02:56.719818 (+ 876us) server_negotiation.cc:206] Beginning negotiation
0806 05:02:56.719830 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 05:02:56.719872 (+ 42us) server_negotiation.cc:407] Connection header received
0806 05:02:56.721555 (+ 1683us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 05:02:56.721576 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 05:02:56.721952 (+ 376us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 05:02:56.722286 (+ 334us) server_negotiation.cc:226] Negotiated authn=SASL
0806 05:02:56.724123 (+ 1837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:02:56.725499 (+ 1376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:02:56.727286 (+ 1787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:02:56.728056 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:02:56.728205 (+ 149us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:02:56.733493 (+ 5288us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 05:02:56.733500 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 05:02:56.733514 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 05:02:56.733548 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0806 05:02:56.736831 (+ 3283us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 05:02:56.737665 (+ 834us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 05:02:56.737674 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 05:02:56.737680 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 05:02:56.737861 (+ 181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 05:02:56.738329 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 05:02:56.738336 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 05:02:56.738343 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 05:02:56.738883 (+ 540us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 05:02:56.739216 (+ 333us) server_negotiation.cc:1035] Waiting for connection context
0806 05:02:56.739809 (+ 593us) server_negotiation.cc:299] Negotiation successful
0806 05:02:56.740119 (+ 310us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":676,"thread_start_us":432,"threads_started":1}
I20260806 05:02:56.742496 9474 raft_consensus.cc:348] T 00000000000000000000000000000000 P 896454a20ba244ab83d5d0181f4f305e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "896454a20ba244ab83d5d0181f4f305e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44765 } }
I20260806 05:02:56.742879 9474 raft_consensus.cc:374] T 00000000000000000000000000000000 P 896454a20ba244ab83d5d0181f4f305e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 05:02:56.743009 9474 raft_consensus.cc:729] T 00000000000000000000000000000000 P 896454a20ba244ab83d5d0181f4f305e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 896454a20ba244ab83d5d0181f4f305e, State: Initialized, Role: FOLLOWER
I20260806 05:02:56.743702 9474 consensus_queue.cc:260] T 00000000000000000000000000000000 P 896454a20ba244ab83d5d0181f4f305e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "896454a20ba244ab83d5d0181f4f305e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44765 } }
I20260806 05:02:56.744001 9474 raft_consensus.cc:388] T 00000000000000000000000000000000 P 896454a20ba244ab83d5d0181f4f305e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 05:02:56.744168 9474 raft_consensus.cc:482] T 00000000000000000000000000000000 P 896454a20ba244ab83d5d0181f4f305e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 05:02:56.744372 9474 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 896454a20ba244ab83d5d0181f4f305e [term 0 FOLLOWER]: Advancing to term 1
I20260806 05:02:56.745121 9601 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 05:02:56.747681 9474 raft_consensus.cc:504] T 00000000000000000000000000000000 P 896454a20ba244ab83d5d0181f4f305e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "896454a20ba244ab83d5d0181f4f305e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44765 } }
I20260806 05:02:56.748294 9474 leader_election.cc:302] T 00000000000000000000000000000000 P 896454a20ba244ab83d5d0181f4f305e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 896454a20ba244ab83d5d0181f4f305e; no voters:
I20260806 05:02:56.749046 9474 leader_election.cc:288] T 00000000000000000000000000000000 P 896454a20ba244ab83d5d0181f4f305e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 05:02:56.749171 9483 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 896454a20ba244ab83d5d0181f4f305e [term 1 FOLLOWER]: Leader election won for term 1
I20260806 05:02:56.749926 9483 raft_consensus.cc:686] T 00000000000000000000000000000000 P 896454a20ba244ab83d5d0181f4f305e [term 1 LEADER]: Becoming Leader. State: Replica: 896454a20ba244ab83d5d0181f4f305e, State: Running, Role: LEADER
I20260806 05:02:56.750587 9483 consensus_queue.cc:237] T 00000000000000000000000000000000 P 896454a20ba244ab83d5d0181f4f305e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "896454a20ba244ab83d5d0181f4f305e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44765 } }
I20260806 05:02:56.752308 9474 sys_catalog.cc:564] T 00000000000000000000000000000000 P 896454a20ba244ab83d5d0181f4f305e [sys.catalog]: configured and running, proceeding with master startup.
I20260806 05:02:56.755965 9489 sys_catalog.cc:455] T 00000000000000000000000000000000 P 896454a20ba244ab83d5d0181f4f305e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "896454a20ba244ab83d5d0181f4f305e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "896454a20ba244ab83d5d0181f4f305e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44765 } } }
I20260806 05:02:56.755877 9488 sys_catalog.cc:455] T 00000000000000000000000000000000 P 896454a20ba244ab83d5d0181f4f305e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 896454a20ba244ab83d5d0181f4f305e. Latest consensus state: current_term: 1 leader_uuid: "896454a20ba244ab83d5d0181f4f305e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "896454a20ba244ab83d5d0181f4f305e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44765 } } }
I20260806 05:02:56.756386 9489 sys_catalog.cc:458] T 00000000000000000000000000000000 P 896454a20ba244ab83d5d0181f4f305e [sys.catalog]: This master's current role is: LEADER
I20260806 05:02:56.756423 9488 sys_catalog.cc:458] T 00000000000000000000000000000000 P 896454a20ba244ab83d5d0181f4f305e [sys.catalog]: This master's current role is: LEADER
I20260806 05:02:56.760489 9500 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 05:02:56.766525 9500 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 05:02:56.770337 9479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:02:56.751096 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.93:47641 (local address 127.0.0.1:44765)
0806 05:02:56.751247 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0806 05:02:56.751252 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 05:02:56.751568 (+ 316us) server_negotiation.cc:407] Connection header received
0806 05:02:56.751984 (+ 416us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 05:02:56.751988 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 05:02:56.752074 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 05:02:56.752199 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0806 05:02:56.754260 (+ 2061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:02:56.755612 (+ 1352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:02:56.757914 (+ 2302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:02:56.758811 (+ 897us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:02:56.758983 (+ 172us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:02:56.762098 (+ 3115us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 05:02:56.762111 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 05:02:56.762119 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 05:02:56.762178 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0806 05:02:56.766500 (+ 4322us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 05:02:56.767510 (+ 1010us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 05:02:56.767515 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 05:02:56.767518 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 05:02:56.767670 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 05:02:56.768318 (+ 648us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 05:02:56.768323 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 05:02:56.768326 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 05:02:56.768788 (+ 462us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 05:02:56.769065 (+ 277us) server_negotiation.cc:1035] Waiting for connection context
0806 05:02:56.769792 (+ 727us) server_negotiation.cc:299] Negotiation successful
0806 05:02:56.769995 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260806 05:02:56.777204 9500 catalog_manager.cc:1347] Generated new cluster ID: 17300357032c434396c67a9dfa18dd54
I20260806 05:02:56.777565 9500 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 05:02:56.791056 9500 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 05:02:56.792351 9500 catalog_manager.cc:1504] Loading token signing keys...
I20260806 05:02:56.807349 9500 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 896454a20ba244ab83d5d0181f4f305e: Generated new TSK 0
I20260806 05:02:56.808333 9500 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 05:02:56.824143 9601 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu pid 9358
I20260806 05:02:56.825511 9468 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 05:02:56.920605 9601 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu with pid 9358
W20260806 05:02:56.946682 9339 connection.cc:535] client connection to 127.0.0.1:44765 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:44765): Connection reset by peer (error 104)
2026-08-06T05:02:56Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (2305 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785992227050689-9601-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 05:02:57 dist-test-slave-x4qh krb5kdc[9530](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 05:02:57 dist-test-slave-x4qh krb5kdc[9530](info): set up 2 sockets
Aug 06 05:02:57 dist-test-slave-x4qh krb5kdc[9530](info): commencing operation
krb5kdc: starting...
W20260806 05:02:58.250989 9601 mini_kdc.cc:121] Time spent starting KDC: real 1.284s user 0.001s sys 0.042s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785992227050689-9601-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785992227050689-9601-0/krb5kdc/test-user.keytab.
Aug 06 05:02:58 dist-test-slave-x4qh krb5kdc[9530](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992578, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T05:02:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T05:02:58Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785992227050689-9601-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785992227050689-9601-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 05:02:58.546932 9601 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
/tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785992227050689-9601-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785992227050689-9601-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785992227050689-9601-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785992227050689-9601-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41209
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785992227050689-9601-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43635
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41209
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785992227050689-9601-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785992227050689-9601-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785992227050689-9601-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 05:02:58.885062 9551 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 05:02:58.885332 9551 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 05:02:58.885525 9551 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 05:02:58.892549 9551 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 05:02:58.892658 9551 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 05:02:58.892807 9551 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 05:02:58.892912 9551 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 05:02:58.892998 9551 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 05:02:58.902472 9551 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43635
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785992227050689-9601-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785992227050689-9601-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41209
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785992227050689-9601-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41209
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785992227050689-9601-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785992227050689-9601-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 03:56:12 UTC on ccycloud.quasar-guijbm.root.comops.site
build id 4569618
ASAN enabled
I20260806 05:02:58.903825 9551 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 05:02:58.905433 9551 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 05:02:58.917510 9561 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 05:02:58.917759 9562 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 05:02:58.917550 9565 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 05:02:58.918500 9551 server_base.cc:1034] running on GCE node
I20260806 05:02:58.919219 9551 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 05:02:58.920558 9551 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 05:02:58.921765 9551 hybrid_clock.cc:648] HybridClock initialized: now 1785992578921721 us; error 33 us; skew 500 ppm
I20260806 05:02:58.922047 9551 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 05:02:58 dist-test-slave-x4qh krb5kdc[9530](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992578, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 05:02:58.925235 9551 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 05:02:58.926684 9551 webserver.cc:460] Webserver started at http://127.0.0.1:37073/ using document root <none> and password file <none>
I20260806 05:02:58.927348 9551 fs_manager.cc:359] Metadata directory not provided
I20260806 05:02:58.927557 9551 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 05:02:58.927879 9551 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 05:02:58.930738 9551 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785992227050689-9601-0/minicluster-data/master-0/data/instance:
uuid: "f157f05d346443adbf78950db8e0745d"
format_stamp: "Formatted at 2026-08-06 05:02:58 on dist-test-slave-x4qh"
server_key: "c6196c85ec6e2826b9ff14aa2ca230fe"
server_key_iv: "15d8c3ef3a32a734ef68b6fb10d6a6f6"
server_key_version: "encryptionkey@0"
I20260806 05:02:58.931622 9551 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785992227050689-9601-0/minicluster-data/master-0/wal/instance:
uuid: "f157f05d346443adbf78950db8e0745d"
format_stamp: "Formatted at 2026-08-06 05:02:58 on dist-test-slave-x4qh"
server_key: "c6196c85ec6e2826b9ff14aa2ca230fe"
server_key_iv: "15d8c3ef3a32a734ef68b6fb10d6a6f6"
server_key_version: "encryptionkey@0"
I20260806 05:02:58.936619 9551 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260806 05:02:58.940069 9578 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 05:02:58.941272 9551 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 05:02:58.941464 9551 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785992227050689-9601-0/minicluster-data/master-0/data,/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785992227050689-9601-0/minicluster-data/master-0/wal
uuid: "f157f05d346443adbf78950db8e0745d"
format_stamp: "Formatted at 2026-08-06 05:02:58 on dist-test-slave-x4qh"
server_key: "c6196c85ec6e2826b9ff14aa2ca230fe"
server_key_iv: "15d8c3ef3a32a734ef68b6fb10d6a6f6"
server_key_version: "encryptionkey@0"
I20260806 05:02:58.941655 9551 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785992227050689-9601-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785992227050689-9601-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785992227050689-9601-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 05:02:58.954073 9551 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 05:02:58.957783 9551 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 05:02:58.958060 9551 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 05:02:58.976517 9551 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41209
I20260806 05:02:58.976567 9657 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41209 every 8 connection(s)
I20260806 05:02:58.977857 9551 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785992227050689-9601-0/minicluster-data/master-0/data/info.pb
I20260806 05:02:58.981612 9658 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 05:02:58.985019 9601 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu as pid 9551
I20260806 05:02:58.985211 9601 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785992227050689-9601-0/minicluster-data/master-0/wal/instance
I20260806 05:02:58.985798 9601 external_mini_cluster.cc:1428] Setting key ec3346afc644020c93d53e8006881ad4
I20260806 05:02:58.994024 9658 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f157f05d346443adbf78950db8e0745d: Bootstrap starting.
I20260806 05:02:58.997730 9658 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f157f05d346443adbf78950db8e0745d: Neither blocks nor log segments found. Creating new log.
I20260806 05:02:58.998967 9658 log.cc:824] T 00000000000000000000000000000000 P f157f05d346443adbf78950db8e0745d: Log is configured to *not* fsync() on all Append() calls
I20260806 05:02:58.999147 9663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:02:58.987683 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.93:59067 (local address 127.0.0.1:41209)
0806 05:02:58.988448 (+ 765us) server_negotiation.cc:206] Beginning negotiation
0806 05:02:58.988459 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 05:02:58.988499 (+ 40us) server_negotiation.cc:407] Connection header received
0806 05:02:58.990132 (+ 1633us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 05:02:58.990153 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 05:02:58.990544 (+ 391us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 05:02:58.990839 (+ 295us) server_negotiation.cc:226] Negotiated authn=SASL
0806 05:02:58.991472 (+ 633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:02:58.992770 (+ 1298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:02:58.994652 (+ 1882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:02:58.995314 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:02:58.995548 (+ 234us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:02:58.996875 (+ 1327us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 05:02:58.996882 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 05:02:58.996895 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 05:02:58.997407 (+ 512us) server_negotiation.cc:922] calling sasl_server_start()
0806 05:02:58.997480 (+ 73us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 05:02:58.997506 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 05:02:58.997623 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0806 05:02:58.998012 (+ 389us) server_negotiation.cc:299] Negotiation successful
0806 05:02:58.998152 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":564,"thread_start_us":360,"threads_started":1}
I20260806 05:02:59.001966 9658 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f157f05d346443adbf78950db8e0745d: No bootstrap required, opened a new log
I20260806 05:02:59.002962 9601 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 05:02:59.010663 9658 raft_consensus.cc:348] T 00000000000000000000000000000000 P f157f05d346443adbf78950db8e0745d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f157f05d346443adbf78950db8e0745d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41209 } }
I20260806 05:02:59.011067 9658 raft_consensus.cc:374] T 00000000000000000000000000000000 P f157f05d346443adbf78950db8e0745d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 05:02:59.011178 9658 raft_consensus.cc:729] T 00000000000000000000000000000000 P f157f05d346443adbf78950db8e0745d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f157f05d346443adbf78950db8e0745d, State: Initialized, Role: FOLLOWER
I20260806 05:02:59.011879 9658 consensus_queue.cc:260] T 00000000000000000000000000000000 P f157f05d346443adbf78950db8e0745d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f157f05d346443adbf78950db8e0745d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41209 } }
I20260806 05:02:59.012151 9658 raft_consensus.cc:388] T 00000000000000000000000000000000 P f157f05d346443adbf78950db8e0745d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 05:02:59.012322 9658 raft_consensus.cc:482] T 00000000000000000000000000000000 P f157f05d346443adbf78950db8e0745d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 05:02:59.012519 9658 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f157f05d346443adbf78950db8e0745d [term 0 FOLLOWER]: Advancing to term 1
I20260806 05:02:59.014137 9663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:02:59.007435 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.93:41165 (local address 127.0.0.1:41209)
0806 05:02:59.007632 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0806 05:02:59.007638 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 05:02:59.008101 (+ 463us) server_negotiation.cc:407] Connection header received
0806 05:02:59.008252 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 05:02:59.008258 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 05:02:59.008347 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 05:02:59.008469 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0806 05:02:59.008925 (+ 456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:02:59.010085 (+ 1160us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:02:59.011561 (+ 1476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:02:59.012005 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:02:59.012157 (+ 152us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:02:59.013369 (+ 1212us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 05:02:59.013374 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 05:02:59.013378 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 05:02:59.013397 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0806 05:02:59.013450 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 05:02:59.013469 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 05:02:59.013599 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0806 05:02:59.013814 (+ 215us) server_negotiation.cc:299] Negotiation successful
0806 05:02:59.013867 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260806 05:02:59.015508 9658 raft_consensus.cc:504] T 00000000000000000000000000000000 P f157f05d346443adbf78950db8e0745d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f157f05d346443adbf78950db8e0745d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41209 } }
I20260806 05:02:59.016070 9658 leader_election.cc:302] T 00000000000000000000000000000000 P f157f05d346443adbf78950db8e0745d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f157f05d346443adbf78950db8e0745d; no voters:
I20260806 05:02:59.016644 9658 leader_election.cc:288] T 00000000000000000000000000000000 P f157f05d346443adbf78950db8e0745d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 05:02:59.016780 9675 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f157f05d346443adbf78950db8e0745d [term 1 FOLLOWER]: Leader election won for term 1
I20260806 05:02:59.018208 9675 raft_consensus.cc:686] T 00000000000000000000000000000000 P f157f05d346443adbf78950db8e0745d [term 1 LEADER]: Becoming Leader. State: Replica: f157f05d346443adbf78950db8e0745d, State: Running, Role: LEADER
I20260806 05:02:59.018698 9675 consensus_queue.cc:237] T 00000000000000000000000000000000 P f157f05d346443adbf78950db8e0745d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f157f05d346443adbf78950db8e0745d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41209 } }
I20260806 05:02:59.018947 9658 sys_catalog.cc:564] T 00000000000000000000000000000000 P f157f05d346443adbf78950db8e0745d [sys.catalog]: configured and running, proceeding with master startup.
I20260806 05:02:59.021962 9677 sys_catalog.cc:455] T 00000000000000000000000000000000 P f157f05d346443adbf78950db8e0745d [sys.catalog]: SysCatalogTable state changed. Reason: New leader f157f05d346443adbf78950db8e0745d. Latest consensus state: current_term: 1 leader_uuid: "f157f05d346443adbf78950db8e0745d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f157f05d346443adbf78950db8e0745d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41209 } } }
I20260806 05:02:59.022351 9677 sys_catalog.cc:458] T 00000000000000000000000000000000 P f157f05d346443adbf78950db8e0745d [sys.catalog]: This master's current role is: LEADER
I20260806 05:02:59.022886 9676 sys_catalog.cc:455] T 00000000000000000000000000000000 P f157f05d346443adbf78950db8e0745d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f157f05d346443adbf78950db8e0745d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f157f05d346443adbf78950db8e0745d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41209 } } }
I20260806 05:02:59.023315 9676 sys_catalog.cc:458] T 00000000000000000000000000000000 P f157f05d346443adbf78950db8e0745d [sys.catalog]: This master's current role is: LEADER
I20260806 05:02:59.028620 9687 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 05:02:59.032824 9687 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 05:02:59.040530 9687 catalog_manager.cc:1347] Generated new cluster ID: 7a482dcfe8af4e6e9553d8cadfe987e1
I20260806 05:02:59.040647 9687 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 05:02:59.051795 9687 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 05:02:59.053551 9687 catalog_manager.cc:1504] Loading token signing keys...
I20260806 05:02:59.061046 9687 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f157f05d346443adbf78950db8e0745d: Generated new TSK 0
I20260806 05:02:59.062000 9687 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 05:02:59.069440 9601 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu pid 9551
W20260806 05:02:59.071087 9649 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.93:59067
W20260806 05:02:59.071918 9601 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu pid 9551: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260806 05:02:59.072115 9601 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu with pid 9551
W20260806 05:02:59.098802 9523 connection.cc:535] client connection to 127.0.0.1:41209 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:41209): Connection reset by peer (error 104)
2026-08-06T05:02:59Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (2150 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785992227050689-9601-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 05:02:59 dist-test-slave-x4qh krb5kdc[9703](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 05:02:59 dist-test-slave-x4qh krb5kdc[9703](info): set up 2 sockets
Aug 06 05:02:59 dist-test-slave-x4qh krb5kdc[9703](info): commencing operation
krb5kdc: starting...
W20260806 05:03:00.388007 9601 mini_kdc.cc:121] Time spent starting KDC: real 1.270s user 0.003s sys 0.038s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785992227050689-9601-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785992227050689-9601-0/krb5kdc/test-user.keytab.
Aug 06 05:03:00 dist-test-slave-x4qh krb5kdc[9703](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992580, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T05:03:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T05:03:00Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785992227050689-9601-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785992227050689-9601-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 05:03:00.689961 9601 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
/tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785992227050689-9601-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785992227050689-9601-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785992227050689-9601-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785992227050689-9601-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37105
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785992227050689-9601-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:44485
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37105
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785992227050689-9601-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785992227050689-9601-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785992227050689-9601-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 05:03:01.028061 9721 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 05:03:01.028412 9721 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 05:03:01.028589 9721 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 05:03:01.035858 9721 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 05:03:01.036010 9721 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 05:03:01.036108 9721 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 05:03:01.036216 9721 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 05:03:01.036299 9721 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 05:03:01.046103 9721 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:44485
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785992227050689-9601-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785992227050689-9601-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37105
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785992227050689-9601-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37105
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785992227050689-9601-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785992227050689-9601-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 03:56:12 UTC on ccycloud.quasar-guijbm.root.comops.site
build id 4569618
ASAN enabled
I20260806 05:03:01.047461 9721 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 05:03:01.048787 9721 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 05:03:01.060854 9727 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 05:03:01.060937 9731 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 05:03:01.060971 9726 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 05:03:01.061815 9721 server_base.cc:1034] running on GCE node
I20260806 05:03:01.062572 9721 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 05:03:01.064002 9721 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 05:03:01.065237 9721 hybrid_clock.cc:648] HybridClock initialized: now 1785992581065204 us; error 56 us; skew 500 ppm
I20260806 05:03:01.065598 9721 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 05:03:01 dist-test-slave-x4qh krb5kdc[9703](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992581, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 05:03:01.073650 9721 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 05:03:01.075121 9721 webserver.cc:460] Webserver started at http://127.0.0.1:36089/ using document root <none> and password file <none>
I20260806 05:03:01.075887 9721 fs_manager.cc:359] Metadata directory not provided
I20260806 05:03:01.076043 9721 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 05:03:01.076361 9721 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 05:03:01.079424 9721 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785992227050689-9601-0/minicluster-data/master-0/data/instance:
uuid: "92031c73213245c882f08cf95550042a"
format_stamp: "Formatted at 2026-08-06 05:03:01 on dist-test-slave-x4qh"
server_key: "f39c3a4e81ee8560b884955bf6e31f8d"
server_key_iv: "2c5f6c3795fdfd4fed17832a9c294f9c"
server_key_version: "encryptionkey@0"
I20260806 05:03:01.080283 9721 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785992227050689-9601-0/minicluster-data/master-0/wal/instance:
uuid: "92031c73213245c882f08cf95550042a"
format_stamp: "Formatted at 2026-08-06 05:03:01 on dist-test-slave-x4qh"
server_key: "f39c3a4e81ee8560b884955bf6e31f8d"
server_key_iv: "2c5f6c3795fdfd4fed17832a9c294f9c"
server_key_version: "encryptionkey@0"
I20260806 05:03:01.085354 9721 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260806 05:03:01.088698 9740 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 05:03:01.089972 9721 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 05:03:01.090143 9721 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785992227050689-9601-0/minicluster-data/master-0/data,/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785992227050689-9601-0/minicluster-data/master-0/wal
uuid: "92031c73213245c882f08cf95550042a"
format_stamp: "Formatted at 2026-08-06 05:03:01 on dist-test-slave-x4qh"
server_key: "f39c3a4e81ee8560b884955bf6e31f8d"
server_key_iv: "2c5f6c3795fdfd4fed17832a9c294f9c"
server_key_version: "encryptionkey@0"
I20260806 05:03:01.090341 9721 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785992227050689-9601-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785992227050689-9601-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785992227050689-9601-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 05:03:01.096067 9721 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 05:03:01.100387 9721 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 05:03:01.100687 9721 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 05:03:01.119750 9721 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37105
I20260806 05:03:01.119836 9814 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37105 every 8 connection(s)
I20260806 05:03:01.121222 9721 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785992227050689-9601-0/minicluster-data/master-0/data/info.pb
I20260806 05:03:01.124939 9816 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 05:03:01.130779 9601 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu as pid 9721
I20260806 05:03:01.130983 9601 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785992227050689-9601-0/minicluster-data/master-0/wal/instance
I20260806 05:03:01.131515 9601 external_mini_cluster.cc:1428] Setting key d9b61064abc4af4a92aebf71dcc935a7
I20260806 05:03:01.135807 9816 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 92031c73213245c882f08cf95550042a: Bootstrap starting.
I20260806 05:03:01.139477 9820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:03:01.133295 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.93:57485 (local address 127.0.0.1:37105)
0806 05:03:01.134156 (+ 861us) server_negotiation.cc:206] Beginning negotiation
0806 05:03:01.134167 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 05:03:01.134207 (+ 40us) server_negotiation.cc:407] Connection header received
0806 05:03:01.136047 (+ 1840us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 05:03:01.136073 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 05:03:01.136397 (+ 324us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 05:03:01.136704 (+ 307us) server_negotiation.cc:226] Negotiated authn=SASL
0806 05:03:01.137884 (+ 1180us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 05:03:01.137890 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 05:03:01.137902 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 05:03:01.137917 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0806 05:03:01.137974 (+ 57us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 05:03:01.137993 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 05:03:01.138167 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0806 05:03:01.138313 (+ 146us) server_negotiation.cc:299] Negotiation successful
0806 05:03:01.138522 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":642,"thread_start_us":393,"threads_started":1}
I20260806 05:03:01.139622 9816 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 92031c73213245c882f08cf95550042a: Neither blocks nor log segments found. Creating new log.
I20260806 05:03:01.141021 9816 log.cc:824] T 00000000000000000000000000000000 P 92031c73213245c882f08cf95550042a: Log is configured to *not* fsync() on all Append() calls
I20260806 05:03:01.143611 9601 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 05:03:01.144343 9816 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 92031c73213245c882f08cf95550042a: No bootstrap required, opened a new log
I20260806 05:03:01.151558 9820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:03:01.149047 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.93:51785 (local address 127.0.0.1:37105)
0806 05:03:01.149241 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0806 05:03:01.149247 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 05:03:01.149603 (+ 356us) server_negotiation.cc:407] Connection header received
0806 05:03:01.149741 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 05:03:01.149747 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 05:03:01.149813 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 05:03:01.149928 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0806 05:03:01.150869 (+ 941us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 05:03:01.150875 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 05:03:01.150879 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 05:03:01.150898 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0806 05:03:01.150950 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 05:03:01.150974 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 05:03:01.151110 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0806 05:03:01.151157 (+ 47us) server_negotiation.cc:299] Negotiation successful
0806 05:03:01.151229 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260806 05:03:01.154599 9816 raft_consensus.cc:348] T 00000000000000000000000000000000 P 92031c73213245c882f08cf95550042a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "92031c73213245c882f08cf95550042a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37105 } }
I20260806 05:03:01.155005 9816 raft_consensus.cc:374] T 00000000000000000000000000000000 P 92031c73213245c882f08cf95550042a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 05:03:01.155126 9816 raft_consensus.cc:729] T 00000000000000000000000000000000 P 92031c73213245c882f08cf95550042a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 92031c73213245c882f08cf95550042a, State: Initialized, Role: FOLLOWER
I20260806 05:03:01.155771 9816 consensus_queue.cc:260] T 00000000000000000000000000000000 P 92031c73213245c882f08cf95550042a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "92031c73213245c882f08cf95550042a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37105 } }
I20260806 05:03:01.156075 9816 raft_consensus.cc:388] T 00000000000000000000000000000000 P 92031c73213245c882f08cf95550042a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 05:03:01.156287 9816 raft_consensus.cc:482] T 00000000000000000000000000000000 P 92031c73213245c882f08cf95550042a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 05:03:01.156492 9816 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 92031c73213245c882f08cf95550042a [term 0 FOLLOWER]: Advancing to term 1
I20260806 05:03:01.158686 9816 raft_consensus.cc:504] T 00000000000000000000000000000000 P 92031c73213245c882f08cf95550042a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "92031c73213245c882f08cf95550042a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37105 } }
I20260806 05:03:01.159195 9816 leader_election.cc:302] T 00000000000000000000000000000000 P 92031c73213245c882f08cf95550042a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 92031c73213245c882f08cf95550042a; no voters:
I20260806 05:03:01.159794 9816 leader_election.cc:288] T 00000000000000000000000000000000 P 92031c73213245c882f08cf95550042a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 05:03:01.159940 9829 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 92031c73213245c882f08cf95550042a [term 1 FOLLOWER]: Leader election won for term 1
I20260806 05:03:01.160622 9829 raft_consensus.cc:686] T 00000000000000000000000000000000 P 92031c73213245c882f08cf95550042a [term 1 LEADER]: Becoming Leader. State: Replica: 92031c73213245c882f08cf95550042a, State: Running, Role: LEADER
I20260806 05:03:01.161172 9829 consensus_queue.cc:237] T 00000000000000000000000000000000 P 92031c73213245c882f08cf95550042a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "92031c73213245c882f08cf95550042a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37105 } }
I20260806 05:03:01.161899 9816 sys_catalog.cc:564] T 00000000000000000000000000000000 P 92031c73213245c882f08cf95550042a [sys.catalog]: configured and running, proceeding with master startup.
I20260806 05:03:01.164352 9832 sys_catalog.cc:455] T 00000000000000000000000000000000 P 92031c73213245c882f08cf95550042a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "92031c73213245c882f08cf95550042a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "92031c73213245c882f08cf95550042a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37105 } } }
I20260806 05:03:01.164744 9832 sys_catalog.cc:458] T 00000000000000000000000000000000 P 92031c73213245c882f08cf95550042a [sys.catalog]: This master's current role is: LEADER
I20260806 05:03:01.165753 9833 sys_catalog.cc:455] T 00000000000000000000000000000000 P 92031c73213245c882f08cf95550042a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 92031c73213245c882f08cf95550042a. Latest consensus state: current_term: 1 leader_uuid: "92031c73213245c882f08cf95550042a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "92031c73213245c882f08cf95550042a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37105 } } }
I20260806 05:03:01.166043 9833 sys_catalog.cc:458] T 00000000000000000000000000000000 P 92031c73213245c882f08cf95550042a [sys.catalog]: This master's current role is: LEADER
I20260806 05:03:01.167246 9840 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 05:03:01.172767 9840 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 05:03:01.183085 9840 catalog_manager.cc:1347] Generated new cluster ID: 2a66d6d42bc14194babdc7f365a9df54
I20260806 05:03:01.183215 9840 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 05:03:01.195155 9840 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 05:03:01.195582 9840 catalog_manager.cc:1504] Loading token signing keys...
I20260806 05:03:01.202952 9840 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 92031c73213245c882f08cf95550042a: Generated new TSK 0
I20260806 05:03:01.203795 9840 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 05:03:01.212313 9601 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu pid 9721
W20260806 05:03:01.213433 9809 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.93:57485
W20260806 05:03:01.214167 9601 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu pid 9721: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260806 05:03:01.214326 9601 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu with pid 9721
2026-08-06T05:03:01Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (2144 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785992227050689-9601-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 05:03:01 dist-test-slave-x4qh krb5kdc[9863](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 05:03:01 dist-test-slave-x4qh krb5kdc[9863](info): set up 2 sockets
Aug 06 05:03:01 dist-test-slave-x4qh krb5kdc[9863](info): commencing operation
krb5kdc: starting...
W20260806 05:03:02.533010 9601 mini_kdc.cc:121] Time spent starting KDC: real 1.270s user 0.004s sys 0.037s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785992227050689-9601-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785992227050689-9601-0/krb5kdc/test-user.keytab.
Aug 06 05:03:02 dist-test-slave-x4qh krb5kdc[9863](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992582, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T05:03:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T05:03:02Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.96.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.96.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.96.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.126.keytab.
Entry for principal kudu/127.9.96.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.126.keytab.
I20260806 05:03:02.828732 9601 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
/tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785992227050689-9601-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785992227050689-9601-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785992227050689-9601-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785992227050689-9601-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.96.126:36255
--webserver_interface=127.9.96.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.126.keytab
--principal=kudu/127.9.96.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.96.84:45689
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.96.126:36255
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785992227050689-9601-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785992227050689-9601-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785992227050689-9601-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 05:03:03.167729 9882 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 05:03:03.168074 9882 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 05:03:03.168243 9882 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 05:03:03.175433 9882 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 05:03:03.175588 9882 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 05:03:03.175683 9882 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 05:03:03.175798 9882 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 05:03:03.175884 9882 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 05:03:03.185871 9882 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.96.84:45689
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785992227050689-9601-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785992227050689-9601-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.96.126:36255
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.126.keytab
--principal=kudu/127.9.96.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.96.126:36255
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785992227050689-9601-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.96.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785992227050689-9601-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 03:56:12 UTC on ccycloud.quasar-guijbm.root.comops.site
build id 4569618
ASAN enabled
I20260806 05:03:03.187126 9882 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 05:03:03.188500 9882 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 05:03:03.200224 9895 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 05:03:03.200275 9900 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 05:03:03.200312 9896 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 05:03:03.201273 9882 server_base.cc:1034] running on GCE node
I20260806 05:03:03.202040 9882 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 05:03:03.203259 9882 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 05:03:03.204468 9882 hybrid_clock.cc:648] HybridClock initialized: now 1785992583204418 us; error 64 us; skew 500 ppm
I20260806 05:03:03.204793 9882 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 05:03:03 dist-test-slave-x4qh krb5kdc[9863](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992583, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.96.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 05:03:03.207912 9882 init.cc:377] Logged in from keytab as kudu/127.9.96.126@KRBTEST.COM (short username kudu)
I20260806 05:03:03.209267 9882 webserver.cc:460] Webserver started at http://127.9.96.126:44625/ using document root <none> and password file <none>
I20260806 05:03:03.209928 9882 fs_manager.cc:359] Metadata directory not provided
I20260806 05:03:03.210069 9882 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 05:03:03.210364 9882 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 05:03:03.213214 9882 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785992227050689-9601-0/minicluster-data/master-0/data/instance:
uuid: "f552cf434af045ea950610ddb437e022"
format_stamp: "Formatted at 2026-08-06 05:03:03 on dist-test-slave-x4qh"
server_key: "7958a741fa305ecd4eda299284ea734f"
server_key_iv: "cfdfb09afe999ef69746727f8122956d"
server_key_version: "encryptionkey@0"
I20260806 05:03:03.213976 9882 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785992227050689-9601-0/minicluster-data/master-0/wal/instance:
uuid: "f552cf434af045ea950610ddb437e022"
format_stamp: "Formatted at 2026-08-06 05:03:03 on dist-test-slave-x4qh"
server_key: "7958a741fa305ecd4eda299284ea734f"
server_key_iv: "cfdfb09afe999ef69746727f8122956d"
server_key_version: "encryptionkey@0"
I20260806 05:03:03.219043 9882 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260806 05:03:03.222337 9912 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 05:03:03.223618 9882 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 05:03:03.223789 9882 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785992227050689-9601-0/minicluster-data/master-0/data,/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785992227050689-9601-0/minicluster-data/master-0/wal
uuid: "f552cf434af045ea950610ddb437e022"
format_stamp: "Formatted at 2026-08-06 05:03:03 on dist-test-slave-x4qh"
server_key: "7958a741fa305ecd4eda299284ea734f"
server_key_iv: "cfdfb09afe999ef69746727f8122956d"
server_key_version: "encryptionkey@0"
I20260806 05:03:03.223978 9882 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785992227050689-9601-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785992227050689-9601-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785992227050689-9601-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 05:03:03.248412 9882 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 05:03:03.252393 9882 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 05:03:03.252673 9882 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 05:03:03.272006 9882 rpc_server.cc:307] RPC server started. Bound to: 127.9.96.126:36255
I20260806 05:03:03.272060 9996 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.96.126:36255 every 8 connection(s)
I20260806 05:03:03.273411 9882 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785992227050689-9601-0/minicluster-data/master-0/data/info.pb
I20260806 05:03:03.277313 9998 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 05:03:03.278185 9601 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu as pid 9882
I20260806 05:03:03.278410 9601 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785992227050689-9601-0/minicluster-data/master-0/wal/instance
I20260806 05:03:03.279049 9601 external_mini_cluster.cc:1428] Setting key 53728d6bd01a74e764f003b8aec05965
I20260806 05:03:03.292809 9998 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f552cf434af045ea950610ddb437e022: Bootstrap starting.
Aug 06 05:03:03 dist-test-slave-x4qh krb5kdc[9863](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992582, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.96.126@KRBTEST.COM
I20260806 05:03:03.296633 9998 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f552cf434af045ea950610ddb437e022: Neither blocks nor log segments found. Creating new log.
I20260806 05:03:03.298045 9998 log.cc:824] T 00000000000000000000000000000000 P f552cf434af045ea950610ddb437e022: Log is configured to *not* fsync() on all Append() calls
I20260806 05:03:03.301420 9998 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f552cf434af045ea950610ddb437e022: No bootstrap required, opened a new log
I20260806 05:03:03.304363 10002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:03:03.281018 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35069 (local address 127.9.96.126:36255)
0806 05:03:03.281846 (+ 828us) server_negotiation.cc:206] Beginning negotiation
0806 05:03:03.281858 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 05:03:03.281899 (+ 41us) server_negotiation.cc:407] Connection header received
0806 05:03:03.284066 (+ 2167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 05:03:03.284091 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 05:03:03.284485 (+ 394us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 05:03:03.284817 (+ 332us) server_negotiation.cc:226] Negotiated authn=SASL
0806 05:03:03.286833 (+ 2016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:03:03.288169 (+ 1336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:03:03.290092 (+ 1923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:03:03.290781 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:03:03.290921 (+ 140us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:03:03.296124 (+ 5203us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 05:03:03.296134 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 05:03:03.296164 (+ 30us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 05:03:03.296222 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0806 05:03:03.300117 (+ 3895us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 05:03:03.300886 (+ 769us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 05:03:03.300894 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 05:03:03.300901 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 05:03:03.301126 (+ 225us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 05:03:03.301591 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 05:03:03.301598 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 05:03:03.301606 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0806 05:03:03.302125 (+ 519us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 05:03:03.302491 (+ 366us) server_negotiation.cc:1035] Waiting for connection context
0806 05:03:03.302984 (+ 493us) server_negotiation.cc:299] Negotiation successful
0806 05:03:03.303317 (+ 333us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":609,"thread_start_us":405,"threads_started":1}
I20260806 05:03:03.308323 9601 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 05:03:03.310428 9998 raft_consensus.cc:348] T 00000000000000000000000000000000 P f552cf434af045ea950610ddb437e022 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f552cf434af045ea950610ddb437e022" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 36255 } }
I20260806 05:03:03.310911 9998 raft_consensus.cc:374] T 00000000000000000000000000000000 P f552cf434af045ea950610ddb437e022 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 05:03:03.311067 9998 raft_consensus.cc:729] T 00000000000000000000000000000000 P f552cf434af045ea950610ddb437e022 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f552cf434af045ea950610ddb437e022, State: Initialized, Role: FOLLOWER
I20260806 05:03:03.311896 9998 consensus_queue.cc:260] T 00000000000000000000000000000000 P f552cf434af045ea950610ddb437e022 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f552cf434af045ea950610ddb437e022" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 36255 } }
I20260806 05:03:03.312219 9998 raft_consensus.cc:388] T 00000000000000000000000000000000 P f552cf434af045ea950610ddb437e022 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 05:03:03.312388 9998 raft_consensus.cc:482] T 00000000000000000000000000000000 P f552cf434af045ea950610ddb437e022 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 05:03:03.312597 9998 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f552cf434af045ea950610ddb437e022 [term 0 FOLLOWER]: Advancing to term 1
I20260806 05:03:03.315337 9998 raft_consensus.cc:504] T 00000000000000000000000000000000 P f552cf434af045ea950610ddb437e022 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f552cf434af045ea950610ddb437e022" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 36255 } }
I20260806 05:03:03.316008 9998 leader_election.cc:302] T 00000000000000000000000000000000 P f552cf434af045ea950610ddb437e022 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f552cf434af045ea950610ddb437e022; no voters:
I20260806 05:03:03.316741 9998 leader_election.cc:288] T 00000000000000000000000000000000 P f552cf434af045ea950610ddb437e022 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 05:03:03.316854 10009 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f552cf434af045ea950610ddb437e022 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 05:03:03.319636 9998 sys_catalog.cc:564] T 00000000000000000000000000000000 P f552cf434af045ea950610ddb437e022 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 05:03:03.319662 10009 raft_consensus.cc:686] T 00000000000000000000000000000000 P f552cf434af045ea950610ddb437e022 [term 1 LEADER]: Becoming Leader. State: Replica: f552cf434af045ea950610ddb437e022, State: Running, Role: LEADER
I20260806 05:03:03.320379 10009 consensus_queue.cc:237] T 00000000000000000000000000000000 P f552cf434af045ea950610ddb437e022 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f552cf434af045ea950610ddb437e022" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 36255 } }
I20260806 05:03:03.322745 10012 sys_catalog.cc:455] T 00000000000000000000000000000000 P f552cf434af045ea950610ddb437e022 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f552cf434af045ea950610ddb437e022. Latest consensus state: current_term: 1 leader_uuid: "f552cf434af045ea950610ddb437e022" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f552cf434af045ea950610ddb437e022" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 36255 } } }
I20260806 05:03:03.323112 10012 sys_catalog.cc:458] T 00000000000000000000000000000000 P f552cf434af045ea950610ddb437e022 [sys.catalog]: This master's current role is: LEADER
I20260806 05:03:03.326010 10010 sys_catalog.cc:455] T 00000000000000000000000000000000 P f552cf434af045ea950610ddb437e022 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f552cf434af045ea950610ddb437e022" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f552cf434af045ea950610ddb437e022" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 36255 } } }
I20260806 05:03:03.328743 10010 sys_catalog.cc:458] T 00000000000000000000000000000000 P f552cf434af045ea950610ddb437e022 [sys.catalog]: This master's current role is: LEADER
I20260806 05:03:03.335246 10002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:03:03.312913 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45445 (local address 127.9.96.126:36255)
0806 05:03:03.313085 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0806 05:03:03.313089 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 05:03:03.313613 (+ 524us) server_negotiation.cc:407] Connection header received
0806 05:03:03.313810 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 05:03:03.313815 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 05:03:03.313896 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 05:03:03.314009 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0806 05:03:03.315495 (+ 1486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:03:03.316376 (+ 881us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:03:03.318101 (+ 1725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:03:03.318582 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:03:03.318659 (+ 77us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:03:03.321651 (+ 2992us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 05:03:03.321663 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 05:03:03.321669 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 05:03:03.321707 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0806 05:03:03.328138 (+ 6431us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 05:03:03.329368 (+ 1230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 05:03:03.329378 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 05:03:03.329386 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0806 05:03:03.329587 (+ 201us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 05:03:03.330407 (+ 820us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 05:03:03.330416 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 05:03:03.330423 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 05:03:03.330996 (+ 573us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 05:03:03.331348 (+ 352us) server_negotiation.cc:1035] Waiting for connection context
0806 05:03:03.334564 (+ 3216us) server_negotiation.cc:299] Negotiation successful
0806 05:03:03.334828 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260806 05:03:03.339970 10017 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 05:03:03.345592 10017 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 05:03:03.354809 10017 catalog_manager.cc:1347] Generated new cluster ID: 63d2390931994f7c999a00d5c93fe5df
I20260806 05:03:03.354936 10017 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 05:03:03.381382 10017 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 05:03:03.382647 10017 catalog_manager.cc:1504] Loading token signing keys...
I20260806 05:03:03.393777 10017 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f552cf434af045ea950610ddb437e022: Generated new TSK 0
I20260806 05:03:03.394568 10017 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 05:03:03.406250 9601 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu pid 9882
I20260806 05:03:03.407864 9988 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 05:03:03.499529 9601 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu with pid 9882
W20260806 05:03:03.528133 9858 connection.cc:535] client connection to 127.9.96.126:36255 recv error: Network error: failed to read from TLS socket (remote: 127.9.96.126:36255): Connection reset by peer (error 104)
2026-08-06T05:03:03Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (2284 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785992227050689-9601-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 05:03:03 dist-test-slave-x4qh krb5kdc[10038](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 05:03:03 dist-test-slave-x4qh krb5kdc[10038](info): set up 2 sockets
Aug 06 05:03:03 dist-test-slave-x4qh krb5kdc[10038](info): commencing operation
krb5kdc: starting...
W20260806 05:03:04.828531 9601 mini_kdc.cc:121] Time spent starting KDC: real 1.282s user 0.000s sys 0.040s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785992227050689-9601-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785992227050689-9601-0/krb5kdc/test-user.keytab.
Aug 06 05:03:05 dist-test-slave-x4qh krb5kdc[10038](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992585, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T05:03:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T05:03:05Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.96.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.96.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.96.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.126.keytab.
Entry for principal kudu/127.9.96.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.126.keytab.
I20260806 05:03:05.131026 9601 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
/tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785992227050689-9601-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785992227050689-9601-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785992227050689-9601-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785992227050689-9601-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.96.126:43659
--webserver_interface=127.9.96.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.126.keytab
--principal=kudu/127.9.96.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.96.84:33715
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.96.126:43659
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785992227050689-9601-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785992227050689-9601-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785992227050689-9601-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 05:03:05.465391 10060 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 05:03:05.465682 10060 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 05:03:05.465870 10060 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 05:03:05.473054 10060 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 05:03:05.473151 10060 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 05:03:05.473233 10060 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 05:03:05.473328 10060 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 05:03:05.473412 10060 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 05:03:05.482923 10060 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.96.84:33715
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785992227050689-9601-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785992227050689-9601-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.96.126:43659
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.126.keytab
--principal=kudu/127.9.96.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.96.126:43659
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785992227050689-9601-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.96.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785992227050689-9601-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 03:56:12 UTC on ccycloud.quasar-guijbm.root.comops.site
build id 4569618
ASAN enabled
I20260806 05:03:05.484197 10060 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 05:03:05.485486 10060 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 05:03:05.497092 10067 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 05:03:05.497126 10072 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 05:03:05.497188 10068 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 05:03:05.498747 10060 server_base.cc:1034] running on GCE node
I20260806 05:03:05.499572 10060 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 05:03:05.500900 10060 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 05:03:05.502142 10060 hybrid_clock.cc:648] HybridClock initialized: now 1785992585502100 us; error 53 us; skew 500 ppm
I20260806 05:03:05.502491 10060 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 05:03:05 dist-test-slave-x4qh krb5kdc[10038](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992585, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.96.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 05:03:05.505667 10060 init.cc:377] Logged in from keytab as kudu/127.9.96.126@KRBTEST.COM (short username kudu)
I20260806 05:03:05.507762 10060 webserver.cc:460] Webserver started at http://127.9.96.126:43693/ using document root <none> and password file <none>
I20260806 05:03:05.508823 10060 fs_manager.cc:359] Metadata directory not provided
I20260806 05:03:05.508980 10060 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 05:03:05.509408 10060 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 05:03:05.513171 10060 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785992227050689-9601-0/minicluster-data/master-0/data/instance:
uuid: "0598fecf972e4d2d8b205cf4b7507509"
format_stamp: "Formatted at 2026-08-06 05:03:05 on dist-test-slave-x4qh"
server_key: "151ba785a5d3a5dbd18c3a50b9e34301"
server_key_iv: "2bf4f421e7522ae499f737e16cf02f47"
server_key_version: "encryptionkey@0"
I20260806 05:03:05.513942 10060 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785992227050689-9601-0/minicluster-data/master-0/wal/instance:
uuid: "0598fecf972e4d2d8b205cf4b7507509"
format_stamp: "Formatted at 2026-08-06 05:03:05 on dist-test-slave-x4qh"
server_key: "151ba785a5d3a5dbd18c3a50b9e34301"
server_key_iv: "2bf4f421e7522ae499f737e16cf02f47"
server_key_version: "encryptionkey@0"
I20260806 05:03:05.519043 10060 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260806 05:03:05.522379 10081 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 05:03:05.523664 10060 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 05:03:05.523845 10060 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785992227050689-9601-0/minicluster-data/master-0/data,/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785992227050689-9601-0/minicluster-data/master-0/wal
uuid: "0598fecf972e4d2d8b205cf4b7507509"
format_stamp: "Formatted at 2026-08-06 05:03:05 on dist-test-slave-x4qh"
server_key: "151ba785a5d3a5dbd18c3a50b9e34301"
server_key_iv: "2bf4f421e7522ae499f737e16cf02f47"
server_key_version: "encryptionkey@0"
I20260806 05:03:05.524039 10060 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785992227050689-9601-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785992227050689-9601-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785992227050689-9601-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 05:03:05.537642 10060 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 05:03:05.541594 10060 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 05:03:05.541880 10060 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 05:03:05.561069 10060 rpc_server.cc:307] RPC server started. Bound to: 127.9.96.126:43659
I20260806 05:03:05.561091 10160 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.96.126:43659 every 8 connection(s)
I20260806 05:03:05.562422 10060 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785992227050689-9601-0/minicluster-data/master-0/data/info.pb
I20260806 05:03:05.566120 10162 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 05:03:05.569036 9601 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu as pid 10060
I20260806 05:03:05.569212 9601 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785992227050689-9601-0/minicluster-data/master-0/wal/instance
I20260806 05:03:05.569669 9601 external_mini_cluster.cc:1428] Setting key 3f318daf8ff98ff1fba6107a93c9692b
I20260806 05:03:05.579296 10162 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0598fecf972e4d2d8b205cf4b7507509: Bootstrap starting.
I20260806 05:03:05.583139 10162 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0598fecf972e4d2d8b205cf4b7507509: Neither blocks nor log segments found. Creating new log.
I20260806 05:03:05.584542 10162 log.cc:824] T 00000000000000000000000000000000 P 0598fecf972e4d2d8b205cf4b7507509: Log is configured to *not* fsync() on all Append() calls
I20260806 05:03:05.584955 10167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:03:05.571498 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35043 (local address 127.9.96.126:43659)
0806 05:03:05.572309 (+ 811us) server_negotiation.cc:206] Beginning negotiation
0806 05:03:05.572324 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 05:03:05.572373 (+ 49us) server_negotiation.cc:407] Connection header received
0806 05:03:05.574780 (+ 2407us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 05:03:05.574810 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 05:03:05.575349 (+ 539us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 05:03:05.575807 (+ 458us) server_negotiation.cc:226] Negotiated authn=SASL
0806 05:03:05.576336 (+ 529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:03:05.578396 (+ 2060us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:03:05.579812 (+ 1416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:03:05.580798 (+ 986us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:03:05.581031 (+ 233us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:03:05.582120 (+ 1089us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 05:03:05.582130 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 05:03:05.582147 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 05:03:05.582833 (+ 686us) server_negotiation.cc:922] calling sasl_server_start()
0806 05:03:05.582934 (+ 101us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 05:03:05.582968 (+ 34us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 05:03:05.583155 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0806 05:03:05.583469 (+ 314us) server_negotiation.cc:299] Negotiation successful
0806 05:03:05.583659 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":560,"thread_start_us":349,"threads_started":1}
I20260806 05:03:05.587824 10162 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0598fecf972e4d2d8b205cf4b7507509: No bootstrap required, opened a new log
I20260806 05:03:05.589138 9601 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 05:03:05.596060 10162 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0598fecf972e4d2d8b205cf4b7507509 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0598fecf972e4d2d8b205cf4b7507509" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 43659 } }
I20260806 05:03:05.596504 10162 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0598fecf972e4d2d8b205cf4b7507509 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 05:03:05.596604 10162 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0598fecf972e4d2d8b205cf4b7507509 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0598fecf972e4d2d8b205cf4b7507509, State: Initialized, Role: FOLLOWER
I20260806 05:03:05.597236 10162 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0598fecf972e4d2d8b205cf4b7507509 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0598fecf972e4d2d8b205cf4b7507509" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 43659 } }
I20260806 05:03:05.597522 10162 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0598fecf972e4d2d8b205cf4b7507509 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 05:03:05.597682 10162 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0598fecf972e4d2d8b205cf4b7507509 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 05:03:05.597879 10162 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0598fecf972e4d2d8b205cf4b7507509 [term 0 FOLLOWER]: Advancing to term 1
I20260806 05:03:05.600292 10167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:03:05.593423 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44047 (local address 127.9.96.126:43659)
0806 05:03:05.593650 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0806 05:03:05.593656 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 05:03:05.594151 (+ 495us) server_negotiation.cc:407] Connection header received
0806 05:03:05.594211 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 05:03:05.594215 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 05:03:05.594274 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 05:03:05.594357 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0806 05:03:05.594979 (+ 622us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:03:05.595798 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:03:05.597661 (+ 1863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:03:05.598214 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:03:05.598306 (+ 92us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:03:05.599514 (+ 1208us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 05:03:05.599519 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 05:03:05.599522 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 05:03:05.599537 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0806 05:03:05.599595 (+ 58us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 05:03:05.599610 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 05:03:05.599695 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0806 05:03:05.599916 (+ 221us) server_negotiation.cc:299] Negotiation successful
0806 05:03:05.599981 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260806 05:03:05.600940 10162 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0598fecf972e4d2d8b205cf4b7507509 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0598fecf972e4d2d8b205cf4b7507509" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 43659 } }
I20260806 05:03:05.601436 10162 leader_election.cc:302] T 00000000000000000000000000000000 P 0598fecf972e4d2d8b205cf4b7507509 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0598fecf972e4d2d8b205cf4b7507509; no voters:
I20260806 05:03:05.602042 10162 leader_election.cc:288] T 00000000000000000000000000000000 P 0598fecf972e4d2d8b205cf4b7507509 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 05:03:05.602185 10178 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0598fecf972e4d2d8b205cf4b7507509 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 05:03:05.602882 10178 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0598fecf972e4d2d8b205cf4b7507509 [term 1 LEADER]: Becoming Leader. State: Replica: 0598fecf972e4d2d8b205cf4b7507509, State: Running, Role: LEADER
I20260806 05:03:05.603478 10178 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0598fecf972e4d2d8b205cf4b7507509 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0598fecf972e4d2d8b205cf4b7507509" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 43659 } }
I20260806 05:03:05.604104 10162 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0598fecf972e4d2d8b205cf4b7507509 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 05:03:05.607507 10179 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0598fecf972e4d2d8b205cf4b7507509 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0598fecf972e4d2d8b205cf4b7507509" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0598fecf972e4d2d8b205cf4b7507509" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 43659 } } }
I20260806 05:03:05.608042 10179 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0598fecf972e4d2d8b205cf4b7507509 [sys.catalog]: This master's current role is: LEADER
I20260806 05:03:05.608311 10180 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0598fecf972e4d2d8b205cf4b7507509 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0598fecf972e4d2d8b205cf4b7507509. Latest consensus state: current_term: 1 leader_uuid: "0598fecf972e4d2d8b205cf4b7507509" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0598fecf972e4d2d8b205cf4b7507509" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 43659 } } }
I20260806 05:03:05.608812 10180 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0598fecf972e4d2d8b205cf4b7507509 [sys.catalog]: This master's current role is: LEADER
I20260806 05:03:05.611903 10190 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 05:03:05.616756 10190 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 05:03:05.630300 10190 catalog_manager.cc:1347] Generated new cluster ID: d84cce2bdf2e45bf8ad7024598ab9f9e
I20260806 05:03:05.630463 10190 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 05:03:05.645550 10190 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 05:03:05.646937 10190 catalog_manager.cc:1504] Loading token signing keys...
I20260806 05:03:05.664916 10190 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0598fecf972e4d2d8b205cf4b7507509: Generated new TSK 0
I20260806 05:03:05.665750 10190 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 05:03:05.677220 9601 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu pid 10060
W20260806 05:03:05.678298 10155 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:35043
W20260806 05:03:05.679052 9601 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu pid 10060: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260806 05:03:05.679199 9601 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu with pid 10060
W20260806 05:03:05.702875 10035 connection.cc:535] client connection to 127.9.96.126:43659 recv error: Network error: failed to read from TLS socket (remote: 127.9.96.126:43659): Connection reset by peer (error 104)
2026-08-06T05:03:05Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (2174 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785992227050689-9601-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 05:03:05 dist-test-slave-x4qh krb5kdc[10212](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 05:03:05 dist-test-slave-x4qh krb5kdc[10212](info): set up 2 sockets
Aug 06 05:03:05 dist-test-slave-x4qh krb5kdc[10212](info): commencing operation
krb5kdc: starting...
W20260806 05:03:07.000825 9601 mini_kdc.cc:121] Time spent starting KDC: real 1.279s user 0.000s sys 0.043s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785992227050689-9601-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785992227050689-9601-0/krb5kdc/test-user.keytab.
Aug 06 05:03:07 dist-test-slave-x4qh krb5kdc[10212](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992587, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T05:03:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T05:03:07Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.96.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.96.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.96.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.126.keytab.
Entry for principal kudu/127.9.96.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.126.keytab.
I20260806 05:03:07.300997 9601 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
/tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785992227050689-9601-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785992227050689-9601-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785992227050689-9601-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785992227050689-9601-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.96.126:33603
--webserver_interface=127.9.96.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.126.keytab
--principal=kudu/127.9.96.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.96.84:33851
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.96.126:33603
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785992227050689-9601-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785992227050689-9601-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785992227050689-9601-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 05:03:07.644140 10236 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 05:03:07.644506 10236 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 05:03:07.644673 10236 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 05:03:07.651940 10236 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 05:03:07.652102 10236 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 05:03:07.652205 10236 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 05:03:07.652333 10236 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 05:03:07.652418 10236 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 05:03:07.662213 10236 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.96.84:33851
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785992227050689-9601-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785992227050689-9601-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.96.126:33603
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.126.keytab
--principal=kudu/127.9.96.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.96.126:33603
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785992227050689-9601-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.96.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785992227050689-9601-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 03:56:12 UTC on ccycloud.quasar-guijbm.root.comops.site
build id 4569618
ASAN enabled
I20260806 05:03:07.663563 10236 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 05:03:07.664951 10236 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 05:03:07.677122 10243 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 05:03:07.677111 10244 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 05:03:07.677114 10248 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 05:03:07.677901 10236 server_base.cc:1034] running on GCE node
I20260806 05:03:07.678632 10236 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 05:03:07.680045 10236 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 05:03:07.681244 10236 hybrid_clock.cc:648] HybridClock initialized: now 1785992587681188 us; error 48 us; skew 500 ppm
I20260806 05:03:07.681674 10236 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 05:03:07 dist-test-slave-x4qh krb5kdc[10212](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992587, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.96.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 05:03:07.690685 10236 init.cc:377] Logged in from keytab as kudu/127.9.96.126@KRBTEST.COM (short username kudu)
I20260806 05:03:07.692788 10236 webserver.cc:460] Webserver started at http://127.9.96.126:46063/ using document root <none> and password file <none>
I20260806 05:03:07.693714 10236 fs_manager.cc:359] Metadata directory not provided
I20260806 05:03:07.693874 10236 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 05:03:07.694265 10236 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 05:03:07.698397 10236 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785992227050689-9601-0/minicluster-data/master-0/data/instance:
uuid: "fccfbbdfaff5431b91293c9a44810808"
format_stamp: "Formatted at 2026-08-06 05:03:07 on dist-test-slave-x4qh"
server_key: "a0bdeb918ebabce33dbe9699958bd24c"
server_key_iv: "68c0e95603368a2cef8de57f4f7261c6"
server_key_version: "encryptionkey@0"
I20260806 05:03:07.699265 10236 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785992227050689-9601-0/minicluster-data/master-0/wal/instance:
uuid: "fccfbbdfaff5431b91293c9a44810808"
format_stamp: "Formatted at 2026-08-06 05:03:07 on dist-test-slave-x4qh"
server_key: "a0bdeb918ebabce33dbe9699958bd24c"
server_key_iv: "68c0e95603368a2cef8de57f4f7261c6"
server_key_version: "encryptionkey@0"
I20260806 05:03:07.706780 10236 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.006s sys 0.000s
I20260806 05:03:07.711563 10256 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 05:03:07.713230 10236 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 05:03:07.713408 10236 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785992227050689-9601-0/minicluster-data/master-0/data,/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785992227050689-9601-0/minicluster-data/master-0/wal
uuid: "fccfbbdfaff5431b91293c9a44810808"
format_stamp: "Formatted at 2026-08-06 05:03:07 on dist-test-slave-x4qh"
server_key: "a0bdeb918ebabce33dbe9699958bd24c"
server_key_iv: "68c0e95603368a2cef8de57f4f7261c6"
server_key_version: "encryptionkey@0"
I20260806 05:03:07.713631 10236 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785992227050689-9601-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785992227050689-9601-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785992227050689-9601-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 05:03:07.723599 10236 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 05:03:07.729651 10236 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 05:03:07.730000 10236 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 05:03:07.752385 10236 rpc_server.cc:307] RPC server started. Bound to: 127.9.96.126:33603
I20260806 05:03:07.752473 10335 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.96.126:33603 every 8 connection(s)
I20260806 05:03:07.754060 10236 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785992227050689-9601-0/minicluster-data/master-0/data/info.pb
I20260806 05:03:07.757827 10336 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 05:03:07.762217 9601 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu as pid 10236
I20260806 05:03:07.762401 9601 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785992227050689-9601-0/minicluster-data/master-0/wal/instance
I20260806 05:03:07.762866 9601 external_mini_cluster.cc:1428] Setting key 8a97c1bba49096c91794bcb3bfa1f866
I20260806 05:03:07.770305 10336 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fccfbbdfaff5431b91293c9a44810808: Bootstrap starting.
I20260806 05:03:07.772050 10340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:03:07.764632 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39997 (local address 127.9.96.126:33603)
0806 05:03:07.765452 (+ 820us) server_negotiation.cc:206] Beginning negotiation
0806 05:03:07.765469 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0806 05:03:07.765522 (+ 53us) server_negotiation.cc:407] Connection header received
0806 05:03:07.768386 (+ 2864us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 05:03:07.768414 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 05:03:07.768812 (+ 398us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 05:03:07.769213 (+ 401us) server_negotiation.cc:226] Negotiated authn=SASL
0806 05:03:07.770171 (+ 958us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 05:03:07.770179 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 05:03:07.770194 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 05:03:07.770212 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0806 05:03:07.770285 (+ 73us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 05:03:07.770330 (+ 45us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 05:03:07.770564 (+ 234us) server_negotiation.cc:1035] Waiting for connection context
0806 05:03:07.770692 (+ 128us) server_negotiation.cc:299] Negotiation successful
0806 05:03:07.770884 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":569,"thread_start_us":317,"threads_started":1}
I20260806 05:03:07.773854 10336 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fccfbbdfaff5431b91293c9a44810808: Neither blocks nor log segments found. Creating new log.
I20260806 05:03:07.775118 10336 log.cc:824] T 00000000000000000000000000000000 P fccfbbdfaff5431b91293c9a44810808: Log is configured to *not* fsync() on all Append() calls
I20260806 05:03:07.775702 9601 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 05:03:07.777907 10336 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fccfbbdfaff5431b91293c9a44810808: No bootstrap required, opened a new log
I20260806 05:03:07.782832 10340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:03:07.780323 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47309 (local address 127.9.96.126:33603)
0806 05:03:07.780559 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0806 05:03:07.780570 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 05:03:07.780948 (+ 378us) server_negotiation.cc:407] Connection header received
0806 05:03:07.781096 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 05:03:07.781099 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 05:03:07.781144 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 05:03:07.781224 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0806 05:03:07.782180 (+ 956us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 05:03:07.782184 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 05:03:07.782187 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 05:03:07.782201 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0806 05:03:07.782237 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 05:03:07.782251 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 05:03:07.782347 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0806 05:03:07.782498 (+ 151us) server_negotiation.cc:299] Negotiation successful
0806 05:03:07.782540 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260806 05:03:07.786667 10336 raft_consensus.cc:348] T 00000000000000000000000000000000 P fccfbbdfaff5431b91293c9a44810808 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fccfbbdfaff5431b91293c9a44810808" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 33603 } }
I20260806 05:03:07.787051 10336 raft_consensus.cc:374] T 00000000000000000000000000000000 P fccfbbdfaff5431b91293c9a44810808 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 05:03:07.787175 10336 raft_consensus.cc:729] T 00000000000000000000000000000000 P fccfbbdfaff5431b91293c9a44810808 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fccfbbdfaff5431b91293c9a44810808, State: Initialized, Role: FOLLOWER
I20260806 05:03:07.787791 10336 consensus_queue.cc:260] T 00000000000000000000000000000000 P fccfbbdfaff5431b91293c9a44810808 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fccfbbdfaff5431b91293c9a44810808" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 33603 } }
I20260806 05:03:07.788064 10336 raft_consensus.cc:388] T 00000000000000000000000000000000 P fccfbbdfaff5431b91293c9a44810808 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 05:03:07.788214 10336 raft_consensus.cc:482] T 00000000000000000000000000000000 P fccfbbdfaff5431b91293c9a44810808 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 05:03:07.788403 10336 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fccfbbdfaff5431b91293c9a44810808 [term 0 FOLLOWER]: Advancing to term 1
I20260806 05:03:07.791162 10336 raft_consensus.cc:504] T 00000000000000000000000000000000 P fccfbbdfaff5431b91293c9a44810808 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fccfbbdfaff5431b91293c9a44810808" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 33603 } }
I20260806 05:03:07.791706 10336 leader_election.cc:302] T 00000000000000000000000000000000 P fccfbbdfaff5431b91293c9a44810808 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fccfbbdfaff5431b91293c9a44810808; no voters:
I20260806 05:03:07.792331 10336 leader_election.cc:288] T 00000000000000000000000000000000 P fccfbbdfaff5431b91293c9a44810808 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 05:03:07.792466 10349 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fccfbbdfaff5431b91293c9a44810808 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 05:03:07.794155 10349 raft_consensus.cc:686] T 00000000000000000000000000000000 P fccfbbdfaff5431b91293c9a44810808 [term 1 LEADER]: Becoming Leader. State: Replica: fccfbbdfaff5431b91293c9a44810808, State: Running, Role: LEADER
I20260806 05:03:07.794493 10336 sys_catalog.cc:564] T 00000000000000000000000000000000 P fccfbbdfaff5431b91293c9a44810808 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 05:03:07.794663 10349 consensus_queue.cc:237] T 00000000000000000000000000000000 P fccfbbdfaff5431b91293c9a44810808 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fccfbbdfaff5431b91293c9a44810808" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 33603 } }
I20260806 05:03:07.797370 10352 sys_catalog.cc:455] T 00000000000000000000000000000000 P fccfbbdfaff5431b91293c9a44810808 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fccfbbdfaff5431b91293c9a44810808. Latest consensus state: current_term: 1 leader_uuid: "fccfbbdfaff5431b91293c9a44810808" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fccfbbdfaff5431b91293c9a44810808" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 33603 } } }
I20260806 05:03:07.798203 10352 sys_catalog.cc:458] T 00000000000000000000000000000000 P fccfbbdfaff5431b91293c9a44810808 [sys.catalog]: This master's current role is: LEADER
I20260806 05:03:07.798643 10350 sys_catalog.cc:455] T 00000000000000000000000000000000 P fccfbbdfaff5431b91293c9a44810808 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fccfbbdfaff5431b91293c9a44810808" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fccfbbdfaff5431b91293c9a44810808" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 33603 } } }
I20260806 05:03:07.799103 10350 sys_catalog.cc:458] T 00000000000000000000000000000000 P fccfbbdfaff5431b91293c9a44810808 [sys.catalog]: This master's current role is: LEADER
I20260806 05:03:07.799736 10364 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 05:03:07.806205 10364 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 05:03:07.814996 10364 catalog_manager.cc:1347] Generated new cluster ID: 5284a20e310a4be38adabcbc086009ee
I20260806 05:03:07.815107 10364 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 05:03:07.841925 10364 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 05:03:07.842497 10364 catalog_manager.cc:1504] Loading token signing keys...
I20260806 05:03:07.850872 10364 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fccfbbdfaff5431b91293c9a44810808: Generated new TSK 0
I20260806 05:03:07.851933 10364 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 05:03:07.857125 9601 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu pid 10236
W20260806 05:03:07.858243 10330 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:39997
W20260806 05:03:07.858925 9601 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu pid 10236: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260806 05:03:07.859076 9601 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu with pid 10236
2026-08-06T05:03:07Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (2179 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785992227050689-9601-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 05:03:07 dist-test-slave-x4qh krb5kdc[10380](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 05:03:07 dist-test-slave-x4qh krb5kdc[10380](info): set up 2 sockets
Aug 06 05:03:07 dist-test-slave-x4qh krb5kdc[10380](info): commencing operation
krb5kdc: starting...
W20260806 05:03:09.176019 9601 mini_kdc.cc:121] Time spent starting KDC: real 1.275s user 0.007s sys 0.034s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785992227050689-9601-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785992227050689-9601-0/krb5kdc/test-user.keytab.
Aug 06 05:03:09 dist-test-slave-x4qh krb5kdc[10380](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992589, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T05:03:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T05:03:09Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.96.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.96.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.96.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.126.keytab.
Entry for principal kudu/127.9.96.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.126.keytab.
I20260806 05:03:09.483067 9601 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
/tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785992227050689-9601-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785992227050689-9601-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785992227050689-9601-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785992227050689-9601-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.96.126:36499
--webserver_interface=127.9.96.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.126.keytab
--principal=kudu/127.9.96.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.96.84:34323
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.96.126:36499
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785992227050689-9601-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785992227050689-9601-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785992227050689-9601-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 05:03:09.827723 10400 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 05:03:09.828075 10400 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 05:03:09.828248 10400 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 05:03:09.835373 10400 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 05:03:09.835541 10400 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 05:03:09.835645 10400 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 05:03:09.835757 10400 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 05:03:09.835837 10400 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 05:03:09.846019 10400 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.96.84:34323
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785992227050689-9601-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785992227050689-9601-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.96.126:36499
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.126.keytab
--principal=kudu/127.9.96.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.96.126:36499
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785992227050689-9601-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.96.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785992227050689-9601-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 03:56:12 UTC on ccycloud.quasar-guijbm.root.comops.site
build id 4569618
ASAN enabled
I20260806 05:03:09.847299 10400 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 05:03:09.848614 10400 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 05:03:09.860620 10410 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 05:03:09.860625 10408 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 05:03:09.860690 10407 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 05:03:09.861542 10400 server_base.cc:1034] running on GCE node
I20260806 05:03:09.862316 10400 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 05:03:09.863740 10400 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 05:03:09.864944 10400 hybrid_clock.cc:648] HybridClock initialized: now 1785992589864912 us; error 30 us; skew 500 ppm
I20260806 05:03:09.865280 10400 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 05:03:09 dist-test-slave-x4qh krb5kdc[10380](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992589, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.96.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 05:03:09.868470 10400 init.cc:377] Logged in from keytab as kudu/127.9.96.126@KRBTEST.COM (short username kudu)
I20260806 05:03:09.869974 10400 webserver.cc:460] Webserver started at http://127.9.96.126:43407/ using document root <none> and password file <none>
I20260806 05:03:09.870644 10400 fs_manager.cc:359] Metadata directory not provided
I20260806 05:03:09.870795 10400 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 05:03:09.871099 10400 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 05:03:09.874027 10400 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785992227050689-9601-0/minicluster-data/master-0/data/instance:
uuid: "98721d8e055c4683b122a28a9b57b34e"
format_stamp: "Formatted at 2026-08-06 05:03:09 on dist-test-slave-x4qh"
server_key: "b90e7ce6d52e2123e8d027d3bc9a1d2a"
server_key_iv: "5cfc4973ceda7cc5f0b7aeecfb2e4f63"
server_key_version: "encryptionkey@0"
I20260806 05:03:09.874792 10400 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785992227050689-9601-0/minicluster-data/master-0/wal/instance:
uuid: "98721d8e055c4683b122a28a9b57b34e"
format_stamp: "Formatted at 2026-08-06 05:03:09 on dist-test-slave-x4qh"
server_key: "b90e7ce6d52e2123e8d027d3bc9a1d2a"
server_key_iv: "5cfc4973ceda7cc5f0b7aeecfb2e4f63"
server_key_version: "encryptionkey@0"
I20260806 05:03:09.879891 10400 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.007s
I20260806 05:03:09.883188 10422 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 05:03:09.884450 10400 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 05:03:09.884640 10400 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785992227050689-9601-0/minicluster-data/master-0/data,/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785992227050689-9601-0/minicluster-data/master-0/wal
uuid: "98721d8e055c4683b122a28a9b57b34e"
format_stamp: "Formatted at 2026-08-06 05:03:09 on dist-test-slave-x4qh"
server_key: "b90e7ce6d52e2123e8d027d3bc9a1d2a"
server_key_iv: "5cfc4973ceda7cc5f0b7aeecfb2e4f63"
server_key_version: "encryptionkey@0"
I20260806 05:03:09.884843 10400 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785992227050689-9601-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785992227050689-9601-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785992227050689-9601-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 05:03:09.909482 10400 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 05:03:09.914217 10400 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 05:03:09.914551 10400 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 05:03:09.934723 10400 rpc_server.cc:307] RPC server started. Bound to: 127.9.96.126:36499
I20260806 05:03:09.934748 10495 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.96.126:36499 every 8 connection(s)
I20260806 05:03:09.936349 10400 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785992227050689-9601-0/minicluster-data/master-0/data/info.pb
I20260806 05:03:09.940094 10498 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 05:03:09.941556 9601 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu as pid 10400
I20260806 05:03:09.941811 9601 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785992227050689-9601-0/minicluster-data/master-0/wal/instance
I20260806 05:03:09.942325 9601 external_mini_cluster.cc:1428] Setting key 932456ccff040b09c2fa0df996b03700
I20260806 05:03:09.953979 10498 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 98721d8e055c4683b122a28a9b57b34e: Bootstrap starting.
Aug 06 05:03:09 dist-test-slave-x4qh krb5kdc[10380](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992589, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.96.126@KRBTEST.COM
I20260806 05:03:09.957999 10498 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 98721d8e055c4683b122a28a9b57b34e: Neither blocks nor log segments found. Creating new log.
I20260806 05:03:09.959075 10498 log.cc:824] T 00000000000000000000000000000000 P 98721d8e055c4683b122a28a9b57b34e: Log is configured to *not* fsync() on all Append() calls
I20260806 05:03:09.962162 10498 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 98721d8e055c4683b122a28a9b57b34e: No bootstrap required, opened a new log
I20260806 05:03:09.966048 10501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:03:09.944206 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48887 (local address 127.9.96.126:36499)
0806 05:03:09.945144 (+ 938us) server_negotiation.cc:206] Beginning negotiation
0806 05:03:09.945156 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 05:03:09.945204 (+ 48us) server_negotiation.cc:407] Connection header received
0806 05:03:09.946916 (+ 1712us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 05:03:09.946944 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 05:03:09.947346 (+ 402us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 05:03:09.947751 (+ 405us) server_negotiation.cc:226] Negotiated authn=SASL
0806 05:03:09.949651 (+ 1900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:03:09.951122 (+ 1471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:03:09.953036 (+ 1914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:03:09.953681 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:03:09.953829 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:03:09.958695 (+ 4866us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 05:03:09.958703 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 05:03:09.958717 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 05:03:09.958756 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0806 05:03:09.961844 (+ 3088us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 05:03:09.962608 (+ 764us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 05:03:09.962614 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 05:03:09.962620 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 05:03:09.962810 (+ 190us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 05:03:09.963241 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 05:03:09.963245 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 05:03:09.963247 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 05:03:09.963820 (+ 573us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 05:03:09.964168 (+ 348us) server_negotiation.cc:1035] Waiting for connection context
0806 05:03:09.964653 (+ 485us) server_negotiation.cc:299] Negotiation successful
0806 05:03:09.965065 (+ 412us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":698,"thread_start_us":459,"threads_started":1}
I20260806 05:03:09.969962 9601 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 05:03:09.971164 10498 raft_consensus.cc:348] T 00000000000000000000000000000000 P 98721d8e055c4683b122a28a9b57b34e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "98721d8e055c4683b122a28a9b57b34e" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 36499 } }
I20260806 05:03:09.971602 10498 raft_consensus.cc:374] T 00000000000000000000000000000000 P 98721d8e055c4683b122a28a9b57b34e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 05:03:09.971715 10498 raft_consensus.cc:729] T 00000000000000000000000000000000 P 98721d8e055c4683b122a28a9b57b34e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 98721d8e055c4683b122a28a9b57b34e, State: Initialized, Role: FOLLOWER
I20260806 05:03:09.972260 10498 consensus_queue.cc:260] T 00000000000000000000000000000000 P 98721d8e055c4683b122a28a9b57b34e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "98721d8e055c4683b122a28a9b57b34e" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 36499 } }
I20260806 05:03:09.972522 10498 raft_consensus.cc:388] T 00000000000000000000000000000000 P 98721d8e055c4683b122a28a9b57b34e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 05:03:09.972662 10498 raft_consensus.cc:482] T 00000000000000000000000000000000 P 98721d8e055c4683b122a28a9b57b34e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 05:03:09.972847 10498 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 98721d8e055c4683b122a28a9b57b34e [term 0 FOLLOWER]: Advancing to term 1
I20260806 05:03:09.975793 10498 raft_consensus.cc:504] T 00000000000000000000000000000000 P 98721d8e055c4683b122a28a9b57b34e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "98721d8e055c4683b122a28a9b57b34e" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 36499 } }
I20260806 05:03:09.976457 10498 leader_election.cc:302] T 00000000000000000000000000000000 P 98721d8e055c4683b122a28a9b57b34e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 98721d8e055c4683b122a28a9b57b34e; no voters:
I20260806 05:03:09.977121 10498 leader_election.cc:288] T 00000000000000000000000000000000 P 98721d8e055c4683b122a28a9b57b34e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 05:03:09.977174 10512 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 98721d8e055c4683b122a28a9b57b34e [term 1 FOLLOWER]: Leader election won for term 1
I20260806 05:03:09.979207 10512 raft_consensus.cc:686] T 00000000000000000000000000000000 P 98721d8e055c4683b122a28a9b57b34e [term 1 LEADER]: Becoming Leader. State: Replica: 98721d8e055c4683b122a28a9b57b34e, State: Running, Role: LEADER
I20260806 05:03:09.979776 10498 sys_catalog.cc:564] T 00000000000000000000000000000000 P 98721d8e055c4683b122a28a9b57b34e [sys.catalog]: configured and running, proceeding with master startup.
I20260806 05:03:09.979754 10512 consensus_queue.cc:237] T 00000000000000000000000000000000 P 98721d8e055c4683b122a28a9b57b34e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "98721d8e055c4683b122a28a9b57b34e" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 36499 } }
I20260806 05:03:09.989912 10516 sys_catalog.cc:455] T 00000000000000000000000000000000 P 98721d8e055c4683b122a28a9b57b34e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 98721d8e055c4683b122a28a9b57b34e. Latest consensus state: current_term: 1 leader_uuid: "98721d8e055c4683b122a28a9b57b34e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "98721d8e055c4683b122a28a9b57b34e" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 36499 } } }
I20260806 05:03:09.990430 10516 sys_catalog.cc:458] T 00000000000000000000000000000000 P 98721d8e055c4683b122a28a9b57b34e [sys.catalog]: This master's current role is: LEADER
I20260806 05:03:09.991000 10515 sys_catalog.cc:455] T 00000000000000000000000000000000 P 98721d8e055c4683b122a28a9b57b34e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "98721d8e055c4683b122a28a9b57b34e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "98721d8e055c4683b122a28a9b57b34e" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 36499 } } }
I20260806 05:03:09.991325 10515 sys_catalog.cc:458] T 00000000000000000000000000000000 P 98721d8e055c4683b122a28a9b57b34e [sys.catalog]: This master's current role is: LEADER
I20260806 05:03:09.995257 10501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:03:09.974523 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50199 (local address 127.9.96.126:36499)
0806 05:03:09.974727 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0806 05:03:09.974733 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 05:03:09.975070 (+ 337us) server_negotiation.cc:407] Connection header received
0806 05:03:09.975225 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 05:03:09.975229 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 05:03:09.975292 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 05:03:09.975425 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0806 05:03:09.977479 (+ 2054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:03:09.978779 (+ 1300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:03:09.980661 (+ 1882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:03:09.981400 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:03:09.981573 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:03:09.985689 (+ 4116us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 05:03:09.985698 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 05:03:09.985705 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 05:03:09.985759 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0806 05:03:09.990381 (+ 4622us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 05:03:09.991522 (+ 1141us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 05:03:09.991531 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 05:03:09.991537 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 05:03:09.991722 (+ 185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 05:03:09.992308 (+ 586us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 05:03:09.992317 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 05:03:09.992323 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 05:03:09.992818 (+ 495us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 05:03:09.994300 (+ 1482us) server_negotiation.cc:1035] Waiting for connection context
0806 05:03:09.994483 (+ 183us) server_negotiation.cc:299] Negotiation successful
0806 05:03:09.994765 (+ 282us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260806 05:03:10.000435 10526 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 05:03:10.005684 10526 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 05:03:10.014575 10526 catalog_manager.cc:1347] Generated new cluster ID: 5ea6286f6cc0432e80cfc26f58febf22
I20260806 05:03:10.014693 10526 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 05:03:10.047960 10526 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 05:03:10.049878 10526 catalog_manager.cc:1504] Loading token signing keys...
I20260806 05:03:10.063514 10526 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 98721d8e055c4683b122a28a9b57b34e: Generated new TSK 0
I20260806 05:03:10.064319 10526 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 05:03:10.076036 9601 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu pid 10400
I20260806 05:03:10.077054 10491 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 05:03:10.167794 9601 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu with pid 10400
2026-08-06T05:03:10Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (2310 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785992227050689-9601-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 05:03:10 dist-test-slave-x4qh krb5kdc[10542](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 05:03:10 dist-test-slave-x4qh krb5kdc[10542](info): set up 2 sockets
Aug 06 05:03:10 dist-test-slave-x4qh krb5kdc[10542](info): commencing operation
krb5kdc: starting...
W20260806 05:03:11.567047 9601 mini_kdc.cc:121] Time spent starting KDC: real 1.355s user 0.001s sys 0.042s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785992227050689-9601-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785992227050689-9601-0/krb5kdc/test-user.keytab.
Aug 06 05:03:11 dist-test-slave-x4qh krb5kdc[10542](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992591, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T05:03:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T05:03:11Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.96.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.96.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.96.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.126.keytab.
Entry for principal kudu/127.9.96.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.126.keytab.
I20260806 05:03:11.882921 9601 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
/tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785992227050689-9601-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785992227050689-9601-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785992227050689-9601-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785992227050689-9601-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.96.126:45727
--webserver_interface=127.9.96.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.126.keytab
--principal=kudu/127.9.96.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.96.84:35445
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.96.126:45727
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785992227050689-9601-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785992227050689-9601-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785992227050689-9601-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 05:03:12.220978 10563 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 05:03:12.221258 10563 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 05:03:12.221433 10563 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 05:03:12.228479 10563 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 05:03:12.228588 10563 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 05:03:12.228695 10563 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 05:03:12.228808 10563 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 05:03:12.228891 10563 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 05:03:12.238356 10563 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.96.84:35445
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785992227050689-9601-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785992227050689-9601-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.96.126:45727
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.126.keytab
--principal=kudu/127.9.96.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.96.126:45727
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785992227050689-9601-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.96.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785992227050689-9601-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 03:56:12 UTC on ccycloud.quasar-guijbm.root.comops.site
build id 4569618
ASAN enabled
I20260806 05:03:12.239627 10563 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 05:03:12.240996 10563 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 05:03:12.252933 10569 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 05:03:12.252998 10572 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 05:03:12.253031 10570 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 05:03:12.253895 10563 server_base.cc:1034] running on GCE node
I20260806 05:03:12.254709 10563 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 05:03:12.256126 10563 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 05:03:12.257359 10563 hybrid_clock.cc:648] HybridClock initialized: now 1785992592257315 us; error 40 us; skew 500 ppm
I20260806 05:03:12.257714 10563 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 05:03:12 dist-test-slave-x4qh krb5kdc[10542](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992592, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.96.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 05:03:12.260836 10563 init.cc:377] Logged in from keytab as kudu/127.9.96.126@KRBTEST.COM (short username kudu)
I20260806 05:03:12.262257 10563 webserver.cc:460] Webserver started at http://127.9.96.126:33721/ using document root <none> and password file <none>
I20260806 05:03:12.262872 10563 fs_manager.cc:359] Metadata directory not provided
I20260806 05:03:12.262975 10563 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 05:03:12.263217 10563 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 05:03:12.266145 10563 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785992227050689-9601-0/minicluster-data/master-0/data/instance:
uuid: "98b314ce6cd44e05b8835cec31f64ada"
format_stamp: "Formatted at 2026-08-06 05:03:12 on dist-test-slave-x4qh"
server_key: "663d486c31dd9ad8c227e7dbc8c14d12"
server_key_iv: "123de09e2d79a4119ef9c6bc7a20dc8a"
server_key_version: "encryptionkey@0"
I20260806 05:03:12.266840 10563 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785992227050689-9601-0/minicluster-data/master-0/wal/instance:
uuid: "98b314ce6cd44e05b8835cec31f64ada"
format_stamp: "Formatted at 2026-08-06 05:03:12 on dist-test-slave-x4qh"
server_key: "663d486c31dd9ad8c227e7dbc8c14d12"
server_key_iv: "123de09e2d79a4119ef9c6bc7a20dc8a"
server_key_version: "encryptionkey@0"
I20260806 05:03:12.271886 10563 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.008s sys 0.000s
I20260806 05:03:12.275207 10582 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 05:03:12.276496 10563 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 05:03:12.276680 10563 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785992227050689-9601-0/minicluster-data/master-0/data,/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785992227050689-9601-0/minicluster-data/master-0/wal
uuid: "98b314ce6cd44e05b8835cec31f64ada"
format_stamp: "Formatted at 2026-08-06 05:03:12 on dist-test-slave-x4qh"
server_key: "663d486c31dd9ad8c227e7dbc8c14d12"
server_key_iv: "123de09e2d79a4119ef9c6bc7a20dc8a"
server_key_version: "encryptionkey@0"
I20260806 05:03:12.276866 10563 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785992227050689-9601-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785992227050689-9601-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785992227050689-9601-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 05:03:12.296702 10563 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 05:03:12.300680 10563 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 05:03:12.300979 10563 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 05:03:12.320253 10563 rpc_server.cc:307] RPC server started. Bound to: 127.9.96.126:45727
I20260806 05:03:12.320297 10651 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.96.126:45727 every 8 connection(s)
I20260806 05:03:12.321671 10563 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785992227050689-9601-0/minicluster-data/master-0/data/info.pb
I20260806 05:03:12.322829 9601 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu as pid 10563
I20260806 05:03:12.323019 9601 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785992227050689-9601-0/minicluster-data/master-0/wal/instance
I20260806 05:03:12.323629 9601 external_mini_cluster.cc:1428] Setting key 4c1762461bf7b0f2e80dcdf1e2eb6738
I20260806 05:03:12.326638 10652 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 05:03:12.339463 10658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:03:12.325555 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60505 (local address 127.9.96.126:45727)
0806 05:03:12.326430 (+ 875us) server_negotiation.cc:206] Beginning negotiation
0806 05:03:12.326446 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 05:03:12.326498 (+ 52us) server_negotiation.cc:407] Connection header received
0806 05:03:12.328895 (+ 2397us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 05:03:12.328924 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 05:03:12.329424 (+ 500us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 05:03:12.329894 (+ 470us) server_negotiation.cc:226] Negotiated authn=SASL
0806 05:03:12.330434 (+ 540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:03:12.332586 (+ 2152us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:03:12.333942 (+ 1356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:03:12.334965 (+ 1023us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:03:12.335174 (+ 209us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:03:12.336483 (+ 1309us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 05:03:12.336495 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 05:03:12.336515 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 05:03:12.337289 (+ 774us) server_negotiation.cc:922] calling sasl_server_start()
0806 05:03:12.337394 (+ 105us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 05:03:12.337440 (+ 46us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 05:03:12.337582 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0806 05:03:12.337792 (+ 210us) server_negotiation.cc:299] Negotiation successful
0806 05:03:12.338080 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":639,"thread_start_us":359,"threads_started":1}
I20260806 05:03:12.340998 10652 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 98b314ce6cd44e05b8835cec31f64ada: Bootstrap starting.
I20260806 05:03:12.343221 9601 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 05:03:12.344493 10652 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 98b314ce6cd44e05b8835cec31f64ada: Neither blocks nor log segments found. Creating new log.
I20260806 05:03:12.345754 10652 log.cc:824] T 00000000000000000000000000000000 P 98b314ce6cd44e05b8835cec31f64ada: Log is configured to *not* fsync() on all Append() calls
I20260806 05:03:12.348819 10652 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 98b314ce6cd44e05b8835cec31f64ada: No bootstrap required, opened a new log
I20260806 05:03:12.355458 10658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:03:12.348753 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56603 (local address 127.9.96.126:45727)
0806 05:03:12.348978 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0806 05:03:12.348987 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 05:03:12.349555 (+ 568us) server_negotiation.cc:407] Connection header received
0806 05:03:12.349702 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 05:03:12.349708 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 05:03:12.349780 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 05:03:12.349888 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0806 05:03:12.350346 (+ 458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:03:12.351529 (+ 1183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:03:12.352813 (+ 1284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:03:12.353469 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:03:12.353614 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:03:12.354638 (+ 1024us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 05:03:12.354649 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 05:03:12.354654 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 05:03:12.354675 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0806 05:03:12.354724 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 05:03:12.354752 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 05:03:12.354851 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0806 05:03:12.354971 (+ 120us) server_negotiation.cc:299] Negotiation successful
0806 05:03:12.355065 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260806 05:03:12.358421 10652 raft_consensus.cc:348] T 00000000000000000000000000000000 P 98b314ce6cd44e05b8835cec31f64ada [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "98b314ce6cd44e05b8835cec31f64ada" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 45727 } }
I20260806 05:03:12.358796 10652 raft_consensus.cc:374] T 00000000000000000000000000000000 P 98b314ce6cd44e05b8835cec31f64ada [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 05:03:12.358878 10652 raft_consensus.cc:729] T 00000000000000000000000000000000 P 98b314ce6cd44e05b8835cec31f64ada [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 98b314ce6cd44e05b8835cec31f64ada, State: Initialized, Role: FOLLOWER
I20260806 05:03:12.359493 10652 consensus_queue.cc:260] T 00000000000000000000000000000000 P 98b314ce6cd44e05b8835cec31f64ada [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "98b314ce6cd44e05b8835cec31f64ada" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 45727 } }
I20260806 05:03:12.359889 10652 raft_consensus.cc:388] T 00000000000000000000000000000000 P 98b314ce6cd44e05b8835cec31f64ada [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 05:03:12.360095 10652 raft_consensus.cc:482] T 00000000000000000000000000000000 P 98b314ce6cd44e05b8835cec31f64ada [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 05:03:12.360325 10652 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 98b314ce6cd44e05b8835cec31f64ada [term 0 FOLLOWER]: Advancing to term 1
I20260806 05:03:12.362578 10652 raft_consensus.cc:504] T 00000000000000000000000000000000 P 98b314ce6cd44e05b8835cec31f64ada [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "98b314ce6cd44e05b8835cec31f64ada" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 45727 } }
I20260806 05:03:12.363087 10652 leader_election.cc:302] T 00000000000000000000000000000000 P 98b314ce6cd44e05b8835cec31f64ada [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 98b314ce6cd44e05b8835cec31f64ada; no voters:
I20260806 05:03:12.363752 10652 leader_election.cc:288] T 00000000000000000000000000000000 P 98b314ce6cd44e05b8835cec31f64ada [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 05:03:12.363934 10667 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 98b314ce6cd44e05b8835cec31f64ada [term 1 FOLLOWER]: Leader election won for term 1
I20260806 05:03:12.365851 10652 sys_catalog.cc:564] T 00000000000000000000000000000000 P 98b314ce6cd44e05b8835cec31f64ada [sys.catalog]: configured and running, proceeding with master startup.
I20260806 05:03:12.365896 10667 raft_consensus.cc:686] T 00000000000000000000000000000000 P 98b314ce6cd44e05b8835cec31f64ada [term 1 LEADER]: Becoming Leader. State: Replica: 98b314ce6cd44e05b8835cec31f64ada, State: Running, Role: LEADER
I20260806 05:03:12.366482 10667 consensus_queue.cc:237] T 00000000000000000000000000000000 P 98b314ce6cd44e05b8835cec31f64ada [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "98b314ce6cd44e05b8835cec31f64ada" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 45727 } }
I20260806 05:03:12.369786 10668 sys_catalog.cc:455] T 00000000000000000000000000000000 P 98b314ce6cd44e05b8835cec31f64ada [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "98b314ce6cd44e05b8835cec31f64ada" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "98b314ce6cd44e05b8835cec31f64ada" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 45727 } } }
I20260806 05:03:12.370164 10668 sys_catalog.cc:458] T 00000000000000000000000000000000 P 98b314ce6cd44e05b8835cec31f64ada [sys.catalog]: This master's current role is: LEADER
I20260806 05:03:12.369905 10670 sys_catalog.cc:455] T 00000000000000000000000000000000 P 98b314ce6cd44e05b8835cec31f64ada [sys.catalog]: SysCatalogTable state changed. Reason: New leader 98b314ce6cd44e05b8835cec31f64ada. Latest consensus state: current_term: 1 leader_uuid: "98b314ce6cd44e05b8835cec31f64ada" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "98b314ce6cd44e05b8835cec31f64ada" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 45727 } } }
I20260806 05:03:12.370330 10670 sys_catalog.cc:458] T 00000000000000000000000000000000 P 98b314ce6cd44e05b8835cec31f64ada [sys.catalog]: This master's current role is: LEADER
I20260806 05:03:12.375952 10678 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 05:03:12.381300 10678 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 05:03:12.392024 10678 catalog_manager.cc:1347] Generated new cluster ID: 6bdb56f8471c456197316ab1a4969956
I20260806 05:03:12.392136 10678 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 05:03:12.422982 10678 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 05:03:12.424396 10678 catalog_manager.cc:1504] Loading token signing keys...
I20260806 05:03:12.435591 10678 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 98b314ce6cd44e05b8835cec31f64ada: Generated new TSK 0
I20260806 05:03:12.436403 10678 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 05:03:12.447688 9601 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu pid 10563
W20260806 05:03:12.448801 10646 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:60505
W20260806 05:03:12.449496 9601 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu pid 10563: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260806 05:03:12.449646 9601 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu with pid 10563
2026-08-06T05:03:12Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (2283 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785992227050689-9601-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 05:03:12 dist-test-slave-x4qh krb5kdc[10696](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 05:03:12 dist-test-slave-x4qh krb5kdc[10696](info): set up 2 sockets
Aug 06 05:03:12 dist-test-slave-x4qh krb5kdc[10696](info): commencing operation
krb5kdc: starting...
W20260806 05:03:13.801606 9601 mini_kdc.cc:121] Time spent starting KDC: real 1.306s user 0.000s sys 0.041s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785992227050689-9601-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785992227050689-9601-0/krb5kdc/test-user.keytab.
Aug 06 05:03:14 dist-test-slave-x4qh krb5kdc[10696](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992594, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T05:03:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T05:03:14Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.96.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.96.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.96.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.126.keytab.
Entry for principal kudu/127.9.96.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.126.keytab.
I20260806 05:03:14.118613 9601 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
/tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785992227050689-9601-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785992227050689-9601-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785992227050689-9601-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785992227050689-9601-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.96.126:42437
--webserver_interface=127.9.96.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.126.keytab
--principal=kudu/127.9.96.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.96.84:45923
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.96.126:42437
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785992227050689-9601-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785992227050689-9601-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785992227050689-9601-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 05:03:14.454622 10715 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 05:03:14.454897 10715 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 05:03:14.455061 10715 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 05:03:14.462177 10715 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 05:03:14.462273 10715 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 05:03:14.462383 10715 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 05:03:14.462499 10715 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 05:03:14.462577 10715 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 05:03:14.472182 10715 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.96.84:45923
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785992227050689-9601-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785992227050689-9601-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.96.126:42437
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.126.keytab
--principal=kudu/127.9.96.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.96.126:42437
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785992227050689-9601-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.96.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785992227050689-9601-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 03:56:12 UTC on ccycloud.quasar-guijbm.root.comops.site
build id 4569618
ASAN enabled
I20260806 05:03:14.473373 10715 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 05:03:14.474629 10715 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 05:03:14.486545 10724 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 05:03:14.486593 10726 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 05:03:14.486641 10723 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 05:03:14.487520 10715 server_base.cc:1034] running on GCE node
I20260806 05:03:14.488298 10715 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 05:03:14.489717 10715 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 05:03:14.490960 10715 hybrid_clock.cc:648] HybridClock initialized: now 1785992594490918 us; error 51 us; skew 500 ppm
I20260806 05:03:14.491313 10715 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 05:03:14 dist-test-slave-x4qh krb5kdc[10696](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992594, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.96.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 05:03:14.498679 10715 init.cc:377] Logged in from keytab as kudu/127.9.96.126@KRBTEST.COM (short username kudu)
I20260806 05:03:14.500276 10715 webserver.cc:460] Webserver started at http://127.9.96.126:38989/ using document root <none> and password file <none>
I20260806 05:03:14.500988 10715 fs_manager.cc:359] Metadata directory not provided
I20260806 05:03:14.501145 10715 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 05:03:14.501457 10715 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 05:03:14.504536 10715 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785992227050689-9601-0/minicluster-data/master-0/data/instance:
uuid: "121980a06c404dec8f4ee413b81cbc39"
format_stamp: "Formatted at 2026-08-06 05:03:14 on dist-test-slave-x4qh"
server_key: "43a2b58168e9327cce2381046d0d86e2"
server_key_iv: "fd5713f582b60ca62d362c642c41e38d"
server_key_version: "encryptionkey@0"
I20260806 05:03:14.505321 10715 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785992227050689-9601-0/minicluster-data/master-0/wal/instance:
uuid: "121980a06c404dec8f4ee413b81cbc39"
format_stamp: "Formatted at 2026-08-06 05:03:14 on dist-test-slave-x4qh"
server_key: "43a2b58168e9327cce2381046d0d86e2"
server_key_iv: "fd5713f582b60ca62d362c642c41e38d"
server_key_version: "encryptionkey@0"
I20260806 05:03:14.510455 10715 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260806 05:03:14.513803 10734 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 05:03:14.515040 10715 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 05:03:14.515229 10715 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785992227050689-9601-0/minicluster-data/master-0/data,/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785992227050689-9601-0/minicluster-data/master-0/wal
uuid: "121980a06c404dec8f4ee413b81cbc39"
format_stamp: "Formatted at 2026-08-06 05:03:14 on dist-test-slave-x4qh"
server_key: "43a2b58168e9327cce2381046d0d86e2"
server_key_iv: "fd5713f582b60ca62d362c642c41e38d"
server_key_version: "encryptionkey@0"
I20260806 05:03:14.515462 10715 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785992227050689-9601-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785992227050689-9601-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785992227050689-9601-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 05:03:14.521152 10715 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 05:03:14.525553 10715 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 05:03:14.525867 10715 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 05:03:14.544777 10715 rpc_server.cc:307] RPC server started. Bound to: 127.9.96.126:42437
I20260806 05:03:14.544857 10810 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.96.126:42437 every 8 connection(s)
I20260806 05:03:14.546175 10715 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785992227050689-9601-0/minicluster-data/master-0/data/info.pb
I20260806 05:03:14.548599 9601 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu as pid 10715
I20260806 05:03:14.548790 9601 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785992227050689-9601-0/minicluster-data/master-0/wal/instance
I20260806 05:03:14.549232 9601 external_mini_cluster.cc:1428] Setting key 69889fab42c31856e409ab2e4727acc8
I20260806 05:03:14.550092 10811 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 05:03:14.556687 10817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:03:14.550913 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49095 (local address 127.9.96.126:42437)
0806 05:03:14.551663 (+ 750us) server_negotiation.cc:206] Beginning negotiation
0806 05:03:14.551674 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 05:03:14.551713 (+ 39us) server_negotiation.cc:407] Connection header received
0806 05:03:14.553331 (+ 1618us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 05:03:14.553352 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 05:03:14.553673 (+ 321us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 05:03:14.554061 (+ 388us) server_negotiation.cc:226] Negotiated authn=SASL
0806 05:03:14.555104 (+ 1043us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 05:03:14.555112 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 05:03:14.555128 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 05:03:14.555141 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0806 05:03:14.555205 (+ 64us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 05:03:14.555233 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 05:03:14.555503 (+ 270us) server_negotiation.cc:1035] Waiting for connection context
0806 05:03:14.555644 (+ 141us) server_negotiation.cc:299] Negotiation successful
0806 05:03:14.555812 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":549,"thread_start_us":282,"threads_started":1}
I20260806 05:03:14.560209 9601 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 05:03:14.562363 10811 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 121980a06c404dec8f4ee413b81cbc39: Bootstrap starting.
I20260806 05:03:14.565822 10811 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 121980a06c404dec8f4ee413b81cbc39: Neither blocks nor log segments found. Creating new log.
I20260806 05:03:14.566975 10817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:03:14.564578 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39113 (local address 127.9.96.126:42437)
0806 05:03:14.564746 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0806 05:03:14.564751 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 05:03:14.565116 (+ 365us) server_negotiation.cc:407] Connection header received
0806 05:03:14.565231 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 05:03:14.565236 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 05:03:14.565301 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 05:03:14.565409 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0806 05:03:14.566325 (+ 916us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 05:03:14.566332 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 05:03:14.566336 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 05:03:14.566359 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0806 05:03:14.566403 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 05:03:14.566422 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 05:03:14.566537 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0806 05:03:14.566613 (+ 76us) server_negotiation.cc:299] Negotiation successful
0806 05:03:14.566665 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260806 05:03:14.567250 10811 log.cc:824] T 00000000000000000000000000000000 P 121980a06c404dec8f4ee413b81cbc39: Log is configured to *not* fsync() on all Append() calls
I20260806 05:03:14.570780 10811 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 121980a06c404dec8f4ee413b81cbc39: No bootstrap required, opened a new log
I20260806 05:03:14.577874 10811 raft_consensus.cc:348] T 00000000000000000000000000000000 P 121980a06c404dec8f4ee413b81cbc39 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "121980a06c404dec8f4ee413b81cbc39" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 42437 } }
I20260806 05:03:14.578261 10811 raft_consensus.cc:374] T 00000000000000000000000000000000 P 121980a06c404dec8f4ee413b81cbc39 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 05:03:14.578384 10811 raft_consensus.cc:729] T 00000000000000000000000000000000 P 121980a06c404dec8f4ee413b81cbc39 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 121980a06c404dec8f4ee413b81cbc39, State: Initialized, Role: FOLLOWER
I20260806 05:03:14.578931 10811 consensus_queue.cc:260] T 00000000000000000000000000000000 P 121980a06c404dec8f4ee413b81cbc39 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "121980a06c404dec8f4ee413b81cbc39" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 42437 } }
I20260806 05:03:14.579174 10811 raft_consensus.cc:388] T 00000000000000000000000000000000 P 121980a06c404dec8f4ee413b81cbc39 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 05:03:14.579330 10811 raft_consensus.cc:482] T 00000000000000000000000000000000 P 121980a06c404dec8f4ee413b81cbc39 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 05:03:14.579546 10811 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 121980a06c404dec8f4ee413b81cbc39 [term 0 FOLLOWER]: Advancing to term 1
I20260806 05:03:14.581700 10811 raft_consensus.cc:504] T 00000000000000000000000000000000 P 121980a06c404dec8f4ee413b81cbc39 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "121980a06c404dec8f4ee413b81cbc39" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 42437 } }
I20260806 05:03:14.582209 10811 leader_election.cc:302] T 00000000000000000000000000000000 P 121980a06c404dec8f4ee413b81cbc39 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 121980a06c404dec8f4ee413b81cbc39; no voters:
I20260806 05:03:14.582839 10811 leader_election.cc:288] T 00000000000000000000000000000000 P 121980a06c404dec8f4ee413b81cbc39 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 05:03:14.582952 10828 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 121980a06c404dec8f4ee413b81cbc39 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 05:03:14.583854 10828 raft_consensus.cc:686] T 00000000000000000000000000000000 P 121980a06c404dec8f4ee413b81cbc39 [term 1 LEADER]: Becoming Leader. State: Replica: 121980a06c404dec8f4ee413b81cbc39, State: Running, Role: LEADER
I20260806 05:03:14.584841 10828 consensus_queue.cc:237] T 00000000000000000000000000000000 P 121980a06c404dec8f4ee413b81cbc39 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "121980a06c404dec8f4ee413b81cbc39" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 42437 } }
I20260806 05:03:14.585170 10811 sys_catalog.cc:564] T 00000000000000000000000000000000 P 121980a06c404dec8f4ee413b81cbc39 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 05:03:14.587785 10829 sys_catalog.cc:455] T 00000000000000000000000000000000 P 121980a06c404dec8f4ee413b81cbc39 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "121980a06c404dec8f4ee413b81cbc39" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "121980a06c404dec8f4ee413b81cbc39" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 42437 } } }
I20260806 05:03:14.588200 10829 sys_catalog.cc:458] T 00000000000000000000000000000000 P 121980a06c404dec8f4ee413b81cbc39 [sys.catalog]: This master's current role is: LEADER
I20260806 05:03:14.588752 10831 sys_catalog.cc:455] T 00000000000000000000000000000000 P 121980a06c404dec8f4ee413b81cbc39 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 121980a06c404dec8f4ee413b81cbc39. Latest consensus state: current_term: 1 leader_uuid: "121980a06c404dec8f4ee413b81cbc39" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "121980a06c404dec8f4ee413b81cbc39" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 42437 } } }
I20260806 05:03:14.589037 10831 sys_catalog.cc:458] T 00000000000000000000000000000000 P 121980a06c404dec8f4ee413b81cbc39 [sys.catalog]: This master's current role is: LEADER
I20260806 05:03:14.592356 10845 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 05:03:14.598310 10845 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 05:03:14.606564 10845 catalog_manager.cc:1347] Generated new cluster ID: 39c214a43fea49bf9fec3979ef8ca5e3
I20260806 05:03:14.606678 10845 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 05:03:14.633625 10845 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 05:03:14.634191 10845 catalog_manager.cc:1504] Loading token signing keys...
I20260806 05:03:14.648461 10845 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 121980a06c404dec8f4ee413b81cbc39: Generated new TSK 0
I20260806 05:03:14.649432 10845 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 05:03:14.656958 9601 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu pid 10715
W20260806 05:03:14.658006 10804 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:49095
W20260806 05:03:14.658711 9601 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu pid 10715: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260806 05:03:14.658857 9601 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu with pid 10715
2026-08-06T05:03:14Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (2206 ms)
[----------] 18 tests from AuthTokenIssuingTest (39820 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785992227050689-9601-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 05:03:14 dist-test-slave-x4qh krb5kdc[10861](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 05:03:14 dist-test-slave-x4qh krb5kdc[10861](info): set up 2 sockets
Aug 06 05:03:14 dist-test-slave-x4qh krb5kdc[10861](info): commencing operation
krb5kdc: starting...
W20260806 05:03:15.982851 9601 mini_kdc.cc:121] Time spent starting KDC: real 1.280s user 0.005s sys 0.038s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785992227050689-9601-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785992227050689-9601-0/krb5kdc/test-user.keytab.
Aug 06 05:03:16 dist-test-slave-x4qh krb5kdc[10861](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992596, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T05:03:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T05:03:16Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.96.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.96.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.96.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.126.keytab.
Entry for principal kudu/127.9.96.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.126.keytab.
I20260806 05:03:16.284935 9601 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
/tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785992227050689-9601-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785992227050689-9601-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785992227050689-9601-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785992227050689-9601-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.96.126:43597
--webserver_interface=127.9.96.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.126.keytab
--principal=kudu/127.9.96.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.96.84:33169
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.96.126:43597,127.9.96.125:39223,127.9.96.124:39387
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785992227050689-9601-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785992227050689-9601-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785992227050689-9601-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 05:03:16.627192 10885 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260806 05:03:16.627584 10885 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 05:03:16.627715 10885 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 05:03:16.627868 10885 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 05:03:16.635100 10885 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 05:03:16.635190 10885 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 05:03:16.635309 10885 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 05:03:16.635408 10885 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 05:03:16.635502 10885 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 05:03:16.645512 10885 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.96.84:33169
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785992227050689-9601-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785992227050689-9601-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.96.126:43597,127.9.96.125:39223,127.9.96.124:39387
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.126.keytab
--principal=kudu/127.9.96.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.96.126:43597
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785992227050689-9601-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.96.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785992227050689-9601-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 03:56:12 UTC on ccycloud.quasar-guijbm.root.comops.site
build id 4569618
ASAN enabled
I20260806 05:03:16.646834 10885 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 05:03:16.648162 10885 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 05:03:16.660857 10896 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 05:03:16.660992 10893 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 05:03:16.660990 10894 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 05:03:16.662223 10885 server_base.cc:1034] running on GCE node
I20260806 05:03:16.663077 10885 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 05:03:16.664510 10885 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 05:03:16.665746 10885 hybrid_clock.cc:648] HybridClock initialized: now 1785992596665676 us; error 43 us; skew 500 ppm
I20260806 05:03:16.666090 10885 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 05:03:16 dist-test-slave-x4qh krb5kdc[10861](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992596, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.96.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 05:03:16.669391 10885 init.cc:377] Logged in from keytab as kudu/127.9.96.126@KRBTEST.COM (short username kudu)
I20260806 05:03:16.671051 10885 webserver.cc:460] Webserver started at http://127.9.96.126:34553/ using document root <none> and password file <none>
I20260806 05:03:16.671896 10885 fs_manager.cc:359] Metadata directory not provided
I20260806 05:03:16.672031 10885 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 05:03:16.672358 10885 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 05:03:16.675432 10885 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785992227050689-9601-0/minicluster-data/master-0/data/instance:
uuid: "800e308d265e4a9597974f5ca7ba342b"
format_stamp: "Formatted at 2026-08-06 05:03:16 on dist-test-slave-x4qh"
server_key: "80dfccdf8abbf3c4e6d90c407594f1b3"
server_key_iv: "4b9fe4ffd5e397d441ad0aefbefbab4f"
server_key_version: "encryptionkey@0"
I20260806 05:03:16.676239 10885 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785992227050689-9601-0/minicluster-data/master-0/wal/instance:
uuid: "800e308d265e4a9597974f5ca7ba342b"
format_stamp: "Formatted at 2026-08-06 05:03:16 on dist-test-slave-x4qh"
server_key: "80dfccdf8abbf3c4e6d90c407594f1b3"
server_key_iv: "4b9fe4ffd5e397d441ad0aefbefbab4f"
server_key_version: "encryptionkey@0"
I20260806 05:03:16.681226 10885 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.003s
I20260806 05:03:16.684672 10904 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 05:03:16.685925 10885 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 05:03:16.686117 10885 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785992227050689-9601-0/minicluster-data/master-0/data,/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785992227050689-9601-0/minicluster-data/master-0/wal
uuid: "800e308d265e4a9597974f5ca7ba342b"
format_stamp: "Formatted at 2026-08-06 05:03:16 on dist-test-slave-x4qh"
server_key: "80dfccdf8abbf3c4e6d90c407594f1b3"
server_key_iv: "4b9fe4ffd5e397d441ad0aefbefbab4f"
server_key_version: "encryptionkey@0"
I20260806 05:03:16.686340 10885 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785992227050689-9601-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785992227050689-9601-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785992227050689-9601-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 05:03:16.703356 10885 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 05:03:16.707825 10885 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 05:03:16.708106 10885 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 05:03:16.727205 10885 rpc_server.cc:307] RPC server started. Bound to: 127.9.96.126:43597
I20260806 05:03:16.727223 10981 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.96.126:43597 every 8 connection(s)
I20260806 05:03:16.728648 10885 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785992227050689-9601-0/minicluster-data/master-0/data/info.pb
I20260806 05:03:16.732344 10983 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 05:03:16.734455 9601 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu as pid 10885
I20260806 05:03:16.734684 9601 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785992227050689-9601-0/minicluster-data/master-0/wal/instance
I20260806 05:03:16.735348 9601 external_mini_cluster.cc:1428] Setting key aaf5e6f5a091d9eeccf3266a5fbedb99
I20260806 05:03:16.738871 10983 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.96.126" port: 43597 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 05:03:16 dist-test-slave-x4qh krb5kdc[10861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992596, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.96.126@KRBTEST.COM for kudu/127.9.96.126@KRBTEST.COM
I20260806 05:03:16.763620 10985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:03:16.741876 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:43597 (local address 127.0.0.1:59714)
0806 05:03:16.742732 (+ 856us) negotiation.cc:107] Waiting for socket to connect
0806 05:03:16.742791 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0806 05:03:16.745625 (+ 2834us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 05:03:16.746485 (+ 860us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 05:03:16.746498 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 05:03:16.747295 (+ 797us) client_negotiation.cc:189] Negotiated authn=SASL
0806 05:03:16.747867 (+ 572us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 05:03:16.747895 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 05:03:16.749653 (+ 1758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 05:03:16.749657 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 05:03:16.750717 (+ 1060us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 05:03:16.750729 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 05:03:16.752092 (+ 1363us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 05:03:16.752097 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 05:03:16.752242 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:03:16.752932 (+ 690us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 05:03:16.752954 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0806 05:03:16.755172 (+ 2218us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 05:03:16.759640 (+ 4468us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 05:03:16.759651 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 05:03:16.759661 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0806 05:03:16.760050 (+ 389us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 05:03:16.760520 (+ 470us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 05:03:16.760523 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 05:03:16.760526 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 05:03:16.760686 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 05:03:16.761700 (+ 1014us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 05:03:16.761706 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 05:03:16.761905 (+ 199us) client_negotiation.cc:769] Sending connection context
0806 05:03:16.762120 (+ 215us) client_negotiation.cc:240] Negotiation successful
0806 05:03:16.762509 (+ 389us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":555,"thread_start_us":320,"threads_started":1}
I20260806 05:03:16.763716 10986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:03:16.742914 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59714 (local address 127.9.96.126:43597)
0806 05:03:16.743546 (+ 632us) server_negotiation.cc:206] Beginning negotiation
0806 05:03:16.743551 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 05:03:16.743597 (+ 46us) server_negotiation.cc:407] Connection header received
0806 05:03:16.745995 (+ 2398us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 05:03:16.746001 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 05:03:16.746218 (+ 217us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 05:03:16.746380 (+ 162us) server_negotiation.cc:226] Negotiated authn=SASL
0806 05:03:16.748095 (+ 1715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:03:16.749481 (+ 1386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:03:16.750870 (+ 1389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:03:16.751876 (+ 1006us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:03:16.752071 (+ 195us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:03:16.755443 (+ 3372us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 05:03:16.755456 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 05:03:16.755475 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 05:03:16.755516 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0806 05:03:16.759512 (+ 3996us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 05:03:16.760193 (+ 681us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 05:03:16.760205 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 05:03:16.760211 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 05:03:16.760406 (+ 195us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 05:03:16.760815 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 05:03:16.760823 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 05:03:16.760829 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 05:03:16.761561 (+ 732us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 05:03:16.761770 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0806 05:03:16.762301 (+ 531us) server_negotiation.cc:299] Negotiation successful
0806 05:03:16.762736 (+ 435us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":77,"server-negotiator.queue_time_us":438,"thread_start_us":292,"threads_started":1}
I20260806 05:03:16.766880 10983 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.96.125" port: 39223 } has no permanent_uuid. Determining permanent_uuid...
I20260806 05:03:16.768043 10985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:03:16.767654 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.125:39223 (local address 127.0.0.1:60092)
0806 05:03:16.767832 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0806 05:03:16.767941 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.96.125:39223: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
W20260806 05:03:16.768339 10905 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.9.96.125:39223: connect: Connection refused (error 111)
WARNING: no policy specified for kudu/127.9.96.125@KRBTEST.COM; defaulting to no policy
I20260806 05:03:16.770006 10985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:03:16.769626 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.125:39223 (local address 127.0.0.1:60102)
0806 05:03:16.769804 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0806 05:03:16.769888 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.96.125:39223: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260806 05:03:16.770222 10983 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.96.125:39223: Network error: Client connection negotiation failed: client connection to 127.9.96.125:39223: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.96.125@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.96.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.125.keytab.
Entry for principal kudu/127.9.96.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.125.keytab.
I20260806 05:03:16.810065 9601 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
/tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785992227050689-9601-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785992227050689-9601-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785992227050689-9601-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785992227050689-9601-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.96.125:39223
--webserver_interface=127.9.96.125
--webserver_port=0
--keytab_file=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.125.keytab
--principal=kudu/127.9.96.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.96.84:33169
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.96.126:43597,127.9.96.125:39223,127.9.96.124:39387
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785992227050689-9601-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785992227050689-9601-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785992227050689-9601-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 05:03:16.819531 10983 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.96.125" port: 39223 } attempt: 1
I20260806 05:03:16.820760 10985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:03:16.820306 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.125:39223 (local address 127.0.0.1:60116)
0806 05:03:16.820516 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0806 05:03:16.820612 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.96.125:39223: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":76}
I20260806 05:03:16.821806 10985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:03:16.821445 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.125:39223 (local address 127.0.0.1:60128)
0806 05:03:16.821607 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0806 05:03:16.821671 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.96.125:39223: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260806 05:03:16.822055 10983 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.96.125:39223: Network error: Client connection negotiation failed: client connection to 127.9.96.125:39223: connect: Connection refused (error 111)
I20260806 05:03:16.890340 10983 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.96.125" port: 39223 } attempt: 2
I20260806 05:03:16.891649 10985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:03:16.891086 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.125:39223 (local address 127.0.0.1:60144)
0806 05:03:16.891343 (+ 257us) negotiation.cc:107] Waiting for socket to connect
0806 05:03:16.891508 (+ 165us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.96.125:39223: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
I20260806 05:03:16.892649 10985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:03:16.892286 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.125:39223 (local address 127.0.0.1:60146)
0806 05:03:16.892434 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0806 05:03:16.892525 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.96.125:39223: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260806 05:03:16.892838 10983 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.96.125:39223: Network error: Client connection negotiation failed: client connection to 127.9.96.125:39223: connect: Connection refused (error 111)
I20260806 05:03:16.984133 10983 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.96.125" port: 39223 } attempt: 3
I20260806 05:03:16.985366 10985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:03:16.984918 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.125:39223 (local address 127.0.0.1:60152)
0806 05:03:16.985129 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0806 05:03:16.985227 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.96.125:39223: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":80}
I20260806 05:03:16.986361 10985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:03:16.986005 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.125:39223 (local address 127.0.0.1:60154)
0806 05:03:16.986153 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0806 05:03:16.986225 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.96.125:39223: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260806 05:03:16.986565 10983 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.96.125:39223: Network error: Client connection negotiation failed: client connection to 127.9.96.125:39223: connect: Connection refused (error 111)
I20260806 05:03:17.129837 10983 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.96.125" port: 39223 } attempt: 4
I20260806 05:03:17.131076 10985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:03:17.130524 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.125:39223 (local address 127.0.0.1:60162)
0806 05:03:17.130824 (+ 300us) negotiation.cc:107] Waiting for socket to connect
0806 05:03:17.130936 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.96.125:39223: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":170}
I20260806 05:03:17.132222 10985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:03:17.131896 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.125:39223 (local address 127.0.0.1:60178)
0806 05:03:17.132030 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0806 05:03:17.132110 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.96.125:39223: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260806 05:03:17.132505 10983 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.96.125:39223: Network error: Client connection negotiation failed: client connection to 127.9.96.125:39223: connect: Connection refused (error 111)
W20260806 05:03:17.151126 10993 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260806 05:03:17.151497 10993 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 05:03:17.151660 10993 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 05:03:17.151799 10993 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 05:03:17.158914 10993 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 05:03:17.158991 10993 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 05:03:17.159049 10993 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 05:03:17.159121 10993 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 05:03:17.159222 10993 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 05:03:17.169224 10993 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.96.84:33169
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785992227050689-9601-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785992227050689-9601-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.9.96.126:43597,127.9.96.125:39223,127.9.96.124:39387
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.125.keytab
--principal=kudu/127.9.96.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.96.125:39223
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785992227050689-9601-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.9.96.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785992227050689-9601-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 03:56:12 UTC on ccycloud.quasar-guijbm.root.comops.site
build id 4569618
ASAN enabled
I20260806 05:03:17.170456 10993 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 05:03:17.171887 10993 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 05:03:17.183662 11002 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 05:03:17.183727 11006 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 05:03:17.183689 11004 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 05:03:17.184836 10993 server_base.cc:1034] running on GCE node
I20260806 05:03:17.185566 10993 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 05:03:17.186902 10993 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 05:03:17.188150 10993 hybrid_clock.cc:648] HybridClock initialized: now 1785992597188090 us; error 54 us; skew 500 ppm
I20260806 05:03:17.188478 10993 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 05:03:17 dist-test-slave-x4qh krb5kdc[10861](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992597, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.96.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 05:03:17.191843 10993 init.cc:377] Logged in from keytab as kudu/127.9.96.125@KRBTEST.COM (short username kudu)
I20260806 05:03:17.193253 10993 webserver.cc:460] Webserver started at http://127.9.96.125:35079/ using document root <none> and password file <none>
I20260806 05:03:17.193943 10993 fs_manager.cc:359] Metadata directory not provided
I20260806 05:03:17.194078 10993 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 05:03:17.194410 10993 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 05:03:17.197597 10993 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785992227050689-9601-0/minicluster-data/master-1/data/instance:
uuid: "6e123e02193a4b0cab2fd9734c2cdda1"
format_stamp: "Formatted at 2026-08-06 05:03:17 on dist-test-slave-x4qh"
server_key: "3d52e302181ee08dfa9ee2f437efc6ac"
server_key_iv: "c739c17753c0b24e4ae70681330f1abd"
server_key_version: "encryptionkey@0"
I20260806 05:03:17.198464 10993 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785992227050689-9601-0/minicluster-data/master-1/wal/instance:
uuid: "6e123e02193a4b0cab2fd9734c2cdda1"
format_stamp: "Formatted at 2026-08-06 05:03:17 on dist-test-slave-x4qh"
server_key: "3d52e302181ee08dfa9ee2f437efc6ac"
server_key_iv: "c739c17753c0b24e4ae70681330f1abd"
server_key_version: "encryptionkey@0"
I20260806 05:03:17.203919 10993 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.003s
I20260806 05:03:17.207219 11015 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 05:03:17.208564 10993 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 05:03:17.208774 10993 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785992227050689-9601-0/minicluster-data/master-1/data,/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785992227050689-9601-0/minicluster-data/master-1/wal
uuid: "6e123e02193a4b0cab2fd9734c2cdda1"
format_stamp: "Formatted at 2026-08-06 05:03:17 on dist-test-slave-x4qh"
server_key: "3d52e302181ee08dfa9ee2f437efc6ac"
server_key_iv: "c739c17753c0b24e4ae70681330f1abd"
server_key_version: "encryptionkey@0"
I20260806 05:03:17.208989 10993 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785992227050689-9601-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785992227050689-9601-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785992227050689-9601-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 05:03:17.253942 10993 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 05:03:17.258456 10993 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 05:03:17.258770 10993 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 05:03:17.277652 10993 rpc_server.cc:307] RPC server started. Bound to: 127.9.96.125:39223
I20260806 05:03:17.277673 11082 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.96.125:39223 every 8 connection(s)
I20260806 05:03:17.278960 10993 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785992227050689-9601-0/minicluster-data/master-1/data/info.pb
I20260806 05:03:17.282565 11083 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 05:03:17.282580 9601 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu as pid 10993
I20260806 05:03:17.282769 9601 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785992227050689-9601-0/minicluster-data/master-1/wal/instance
I20260806 05:03:17.283237 9601 external_mini_cluster.cc:1428] Setting key 1778c9283234caa7d0b4c8de1dc5ec86
I20260806 05:03:17.289012 11083 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.96.126" port: 43597 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 05:03:17 dist-test-slave-x4qh krb5kdc[10861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992597, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.96.125@KRBTEST.COM for kudu/127.9.96.126@KRBTEST.COM
I20260806 05:03:17.310946 11087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:03:17.291697 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59726 (local address 127.9.96.126:43597)
0806 05:03:17.292187 (+ 490us) server_negotiation.cc:206] Beginning negotiation
0806 05:03:17.292192 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 05:03:17.293217 (+ 1025us) server_negotiation.cc:407] Connection header received
0806 05:03:17.295043 (+ 1826us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 05:03:17.295049 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 05:03:17.295148 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 05:03:17.295268 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0806 05:03:17.297096 (+ 1828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:03:17.297799 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:03:17.299297 (+ 1498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:03:17.299803 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:03:17.299947 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:03:17.303498 (+ 3551us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 05:03:17.303507 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 05:03:17.303515 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 05:03:17.303579 (+ 64us) server_negotiation.cc:922] calling sasl_server_start()
0806 05:03:17.307553 (+ 3974us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 05:03:17.308263 (+ 710us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 05:03:17.308272 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 05:03:17.308278 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 05:03:17.308445 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 05:03:17.308853 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 05:03:17.308860 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 05:03:17.308866 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 05:03:17.309277 (+ 411us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 05:03:17.309454 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0806 05:03:17.310156 (+ 702us) server_negotiation.cc:299] Negotiation successful
0806 05:03:17.310427 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":382,"thread_start_us":268,"threads_started":1}
I20260806 05:03:17.311539 11088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:03:17.292311 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:43597 (local address 127.0.0.1:59726)
0806 05:03:17.293062 (+ 751us) negotiation.cc:107] Waiting for socket to connect
0806 05:03:17.293104 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0806 05:03:17.294774 (+ 1670us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 05:03:17.295523 (+ 749us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 05:03:17.295531 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 05:03:17.296109 (+ 578us) client_negotiation.cc:189] Negotiated authn=SASL
0806 05:03:17.296832 (+ 723us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 05:03:17.296857 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 05:03:17.297952 (+ 1095us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 05:03:17.297957 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 05:03:17.299146 (+ 1189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 05:03:17.299158 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 05:03:17.300032 (+ 874us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 05:03:17.300039 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 05:03:17.300324 (+ 285us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:03:17.301045 (+ 721us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 05:03:17.301079 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0806 05:03:17.303231 (+ 2152us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 05:03:17.307682 (+ 4451us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 05:03:17.307689 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 05:03:17.307701 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0806 05:03:17.308135 (+ 434us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 05:03:17.308558 (+ 423us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 05:03:17.308566 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 05:03:17.308569 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 05:03:17.308735 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 05:03:17.309407 (+ 672us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 05:03:17.309412 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 05:03:17.309755 (+ 343us) client_negotiation.cc:769] Sending connection context
0806 05:03:17.310018 (+ 263us) client_negotiation.cc:240] Negotiation successful
0806 05:03:17.310391 (+ 373us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":536,"thread_start_us":336,"threads_started":1}
I20260806 05:03:17.313463 11083 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.96.125" port: 39223 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.9.96.124@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.96.124@KRBTEST.COM" created.
Aug 06 05:03:17 dist-test-slave-x4qh krb5kdc[10861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992597, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.96.125@KRBTEST.COM for kudu/127.9.96.125@KRBTEST.COM
I20260806 05:03:17.328634 11088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:03:17.314356 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.125:39223 (local address 127.0.0.1:60182)
0806 05:03:17.314559 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0806 05:03:17.314584 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0806 05:03:17.314694 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 05:03:17.315920 (+ 1226us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 05:03:17.315923 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 05:03:17.316270 (+ 347us) client_negotiation.cc:189] Negotiated authn=SASL
0806 05:03:17.316429 (+ 159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 05:03:17.316437 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 05:03:17.318337 (+ 1900us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 05:03:17.318341 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 05:03:17.319329 (+ 988us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 05:03:17.319342 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 05:03:17.320210 (+ 868us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 05:03:17.320215 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 05:03:17.320411 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:03:17.321076 (+ 665us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 05:03:17.321093 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 05:03:17.323025 (+ 1932us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 05:03:17.326104 (+ 3079us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 05:03:17.326111 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 05:03:17.326115 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 05:03:17.326453 (+ 338us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 05:03:17.326847 (+ 394us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 05:03:17.326851 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 05:03:17.326855 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 05:03:17.326987 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 05:03:17.327643 (+ 656us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 05:03:17.327647 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 05:03:17.327787 (+ 140us) client_negotiation.cc:769] Sending connection context
0806 05:03:17.327939 (+ 152us) client_negotiation.cc:240] Negotiation successful
0806 05:03:17.328209 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":48}
I20260806 05:03:17.328694 11091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:03:17.314591 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60182 (local address 127.9.96.125:39223)
0806 05:03:17.315386 (+ 795us) server_negotiation.cc:206] Beginning negotiation
0806 05:03:17.315397 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 05:03:17.315432 (+ 35us) server_negotiation.cc:407] Connection header received
0806 05:03:17.315492 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 05:03:17.315500 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 05:03:17.315746 (+ 246us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 05:03:17.315921 (+ 175us) server_negotiation.cc:226] Negotiated authn=SASL
0806 05:03:17.316613 (+ 692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:03:17.318213 (+ 1600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:03:17.319515 (+ 1302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:03:17.320051 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:03:17.320153 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:03:17.323236 (+ 3083us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 05:03:17.323247 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 05:03:17.323260 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 05:03:17.323289 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0806 05:03:17.325977 (+ 2688us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 05:03:17.326582 (+ 605us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 05:03:17.326597 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 05:03:17.326599 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 05:03:17.326716 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 05:03:17.327091 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 05:03:17.327098 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 05:03:17.327101 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 05:03:17.327509 (+ 408us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 05:03:17.327718 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0806 05:03:17.328088 (+ 370us) server_negotiation.cc:299] Negotiation successful
0806 05:03:17.328317 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":583,"thread_start_us":425,"threads_started":1}
I20260806 05:03:17.331288 11083 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.96.124" port: 39387 } has no permanent_uuid. Determining permanent_uuid...
I20260806 05:03:17.332544 11088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:03:17.331973 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.124:39387 (local address 127.0.0.1:53758)
0806 05:03:17.332183 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0806 05:03:17.332339 (+ 156us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.96.124:39387: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
W20260806 05:03:17.332845 11016 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.9.96.124:39387: connect: Connection refused (error 111)
I20260806 05:03:17.334560 11088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:03:17.334131 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.124:39387 (local address 127.0.0.1:53766)
0806 05:03:17.334300 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0806 05:03:17.334411 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.96.124:39387: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260806 05:03:17.334800 11083 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.96.124:39387: Network error: Client connection negotiation failed: client connection to 127.9.96.124:39387: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.96.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.124.keytab.
Entry for principal kudu/127.9.96.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.124.keytab.
I20260806 05:03:17.358747 9601 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
/tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785992227050689-9601-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785992227050689-9601-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785992227050689-9601-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785992227050689-9601-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.96.124:39387
--webserver_interface=127.9.96.124
--webserver_port=0
--keytab_file=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.124.keytab
--principal=kudu/127.9.96.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.96.84:33169
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.96.126:43597,127.9.96.125:39223,127.9.96.124:39387
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785992227050689-9601-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785992227050689-9601-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785992227050689-9601-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 05:03:17.384194 11083 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.96.124" port: 39387 } attempt: 1
I20260806 05:03:17.385592 11088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:03:17.384970 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.124:39387 (local address 127.0.0.1:53770)
0806 05:03:17.385231 (+ 261us) negotiation.cc:107] Waiting for socket to connect
0806 05:03:17.385376 (+ 145us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.96.124:39387: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":82}
I20260806 05:03:17.386719 11088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:03:17.386271 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.124:39387 (local address 127.0.0.1:53782)
0806 05:03:17.386449 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0806 05:03:17.386566 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.96.124:39387: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260806 05:03:17.386976 11083 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.96.124:39387: Network error: Client connection negotiation failed: client connection to 127.9.96.124:39387: connect: Connection refused (error 111)
I20260806 05:03:17.431771 10983 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.96.125" port: 39223 } attempt: 5
Aug 06 05:03:17 dist-test-slave-x4qh krb5kdc[10861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992596, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.96.126@KRBTEST.COM for kudu/127.9.96.125@KRBTEST.COM
I20260806 05:03:17.445611 11091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:03:17.432684 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60196 (local address 127.9.96.125:39223)
0806 05:03:17.432861 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0806 05:03:17.432869 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 05:03:17.432906 (+ 37us) server_negotiation.cc:407] Connection header received
0806 05:03:17.433135 (+ 229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 05:03:17.433140 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 05:03:17.433220 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 05:03:17.433332 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0806 05:03:17.434249 (+ 917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:03:17.435092 (+ 843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:03:17.436374 (+ 1282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:03:17.436830 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:03:17.436980 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:03:17.440320 (+ 3340us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 05:03:17.440325 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 05:03:17.440328 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 05:03:17.440361 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0806 05:03:17.442893 (+ 2532us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 05:03:17.443584 (+ 691us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 05:03:17.443592 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 05:03:17.443596 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 05:03:17.443746 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 05:03:17.444172 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 05:03:17.444176 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 05:03:17.444178 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 05:03:17.444474 (+ 296us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 05:03:17.444642 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0806 05:03:17.445073 (+ 431us) server_negotiation.cc:299] Negotiation successful
0806 05:03:17.445289 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260806 05:03:17.445618 10985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:03:17.432594 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.125:39223 (local address 127.0.0.1:60196)
0806 05:03:17.432816 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0806 05:03:17.432837 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0806 05:03:17.432981 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 05:03:17.433367 (+ 386us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 05:03:17.433388 (+ 21us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 05:03:17.433801 (+ 413us) client_negotiation.cc:189] Negotiated authn=SASL
0806 05:03:17.434061 (+ 260us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 05:03:17.434070 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 05:03:17.435230 (+ 1160us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 05:03:17.435237 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 05:03:17.436210 (+ 973us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 05:03:17.436223 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 05:03:17.436993 (+ 770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 05:03:17.436999 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 05:03:17.437196 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:03:17.437848 (+ 652us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 05:03:17.437866 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0806 05:03:17.440122 (+ 2256us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 05:03:17.443026 (+ 2904us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 05:03:17.443033 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 05:03:17.443037 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 05:03:17.443464 (+ 427us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 05:03:17.443864 (+ 400us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 05:03:17.443869 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 05:03:17.443872 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 05:03:17.444053 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 05:03:17.444610 (+ 557us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 05:03:17.444615 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 05:03:17.444778 (+ 163us) client_negotiation.cc:769] Sending connection context
0806 05:03:17.444942 (+ 164us) client_negotiation.cc:240] Negotiation successful
0806 05:03:17.445224 (+ 282us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":75}
I20260806 05:03:17.446981 10983 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.96.124" port: 39387 } has no permanent_uuid. Determining permanent_uuid...
I20260806 05:03:17.447970 10985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:03:17.447583 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.124:39387 (local address 127.0.0.1:53798)
0806 05:03:17.447754 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0806 05:03:17.447829 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.96.124:39387: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
I20260806 05:03:17.448869 10985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:03:17.448557 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.124:39387 (local address 127.0.0.1:53802)
0806 05:03:17.448701 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0806 05:03:17.448767 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.96.124:39387: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260806 05:03:17.449126 10983 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.96.124:39387: Network error: Client connection negotiation failed: client connection to 127.9.96.124:39387: connect: Connection refused (error 111)
I20260806 05:03:17.455219 11083 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.96.124" port: 39387 } attempt: 2
I20260806 05:03:17.456279 11088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:03:17.455827 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.124:39387 (local address 127.0.0.1:53814)
0806 05:03:17.456020 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0806 05:03:17.456117 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.96.124:39387: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
I20260806 05:03:17.457273 11088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:03:17.456913 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.124:39387 (local address 127.0.0.1:53830)
0806 05:03:17.457057 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0806 05:03:17.457138 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.96.124:39387: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260806 05:03:17.457480 11083 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.96.124:39387: Network error: Client connection negotiation failed: client connection to 127.9.96.124:39387: connect: Connection refused (error 111)
I20260806 05:03:17.500330 10983 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.96.124" port: 39387 } attempt: 1
I20260806 05:03:17.501572 10985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:03:17.501018 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.124:39387 (local address 127.0.0.1:53838)
0806 05:03:17.501253 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0806 05:03:17.501367 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.96.124:39387: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":80}
I20260806 05:03:17.502650 10985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:03:17.502250 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.124:39387 (local address 127.0.0.1:53846)
0806 05:03:17.502430 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0806 05:03:17.502517 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.96.124:39387: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
W20260806 05:03:17.502908 10983 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.96.124:39387: Network error: Client connection negotiation failed: client connection to 127.9.96.124:39387: connect: Connection refused (error 111)
I20260806 05:03:17.548712 11083 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.96.124" port: 39387 } attempt: 3
I20260806 05:03:17.549998 11088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:03:17.549464 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.124:39387 (local address 127.0.0.1:53856)
0806 05:03:17.549723 (+ 259us) negotiation.cc:107] Waiting for socket to connect
0806 05:03:17.549824 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.96.124:39387: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":88}
I20260806 05:03:17.551226 11088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:03:17.550750 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.124:39387 (local address 127.0.0.1:53870)
0806 05:03:17.551007 (+ 257us) negotiation.cc:107] Waiting for socket to connect
0806 05:03:17.551094 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.96.124:39387: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":167}
W20260806 05:03:17.551570 11083 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.96.124:39387: Network error: Client connection negotiation failed: client connection to 127.9.96.124:39387: connect: Connection refused (error 111)
I20260806 05:03:17.571118 10983 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.96.124" port: 39387 } attempt: 2
I20260806 05:03:17.572342 10985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:03:17.571758 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.124:39387 (local address 127.0.0.1:53884)
0806 05:03:17.572024 (+ 266us) negotiation.cc:107] Waiting for socket to connect
0806 05:03:17.572143 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.96.124:39387: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":89}
I20260806 05:03:17.573386 10985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:03:17.573009 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.124:39387 (local address 127.0.0.1:53886)
0806 05:03:17.573169 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0806 05:03:17.573255 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.96.124:39387: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260806 05:03:17.573618 10983 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.96.124:39387: Network error: Client connection negotiation failed: client connection to 127.9.96.124:39387: connect: Connection refused (error 111)
I20260806 05:03:17.679881 10983 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.96.124" port: 39387 } attempt: 3
I20260806 05:03:17.681054 10985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:03:17.680508 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.124:39387 (local address 127.0.0.1:53892)
0806 05:03:17.680737 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0806 05:03:17.680856 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.96.124:39387: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
I20260806 05:03:17.682143 10985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:03:17.681737 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.124:39387 (local address 127.0.0.1:53908)
0806 05:03:17.681922 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0806 05:03:17.682012 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.96.124:39387: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
W20260806 05:03:17.682374 10983 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.96.124:39387: Network error: Client connection negotiation failed: client connection to 127.9.96.124:39387: connect: Connection refused (error 111)
I20260806 05:03:17.694798 11083 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.96.124" port: 39387 } attempt: 4
I20260806 05:03:17.695931 11088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:03:17.695457 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.124:39387 (local address 127.0.0.1:53916)
0806 05:03:17.695711 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0806 05:03:17.695790 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.96.124:39387: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":90}
I20260806 05:03:17.696998 11088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:03:17.696683 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.124:39387 (local address 127.0.0.1:53932)
0806 05:03:17.696836 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0806 05:03:17.696902 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.96.124:39387: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260806 05:03:17.697237 11083 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.96.124:39387: Network error: Client connection negotiation failed: client connection to 127.9.96.124:39387: connect: Connection refused (error 111)
W20260806 05:03:17.703364 11098 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260806 05:03:17.703753 11098 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 05:03:17.703881 11098 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 05:03:17.704020 11098 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 05:03:17.711243 11098 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 05:03:17.711366 11098 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 05:03:17.711513 11098 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 05:03:17.711606 11098 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 05:03:17.711716 11098 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 05:03:17.721532 11098 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.96.84:33169
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785992227050689-9601-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785992227050689-9601-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.9.96.126:43597,127.9.96.125:39223,127.9.96.124:39387
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.124.keytab
--principal=kudu/127.9.96.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.96.124:39387
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785992227050689-9601-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.9.96.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785992227050689-9601-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 03:56:12 UTC on ccycloud.quasar-guijbm.root.comops.site
build id 4569618
ASAN enabled
I20260806 05:03:17.722855 11098 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 05:03:17.724187 11098 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 05:03:17.735954 11113 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 05:03:17.735932 11108 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 05:03:17.736413 11107 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 05:03:17.736723 11098 server_base.cc:1034] running on GCE node
I20260806 05:03:17.737462 11098 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 05:03:17.738829 11098 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 05:03:17.740059 11098 hybrid_clock.cc:648] HybridClock initialized: now 1785992597740005 us; error 29 us; skew 500 ppm
I20260806 05:03:17.740348 11098 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 05:03:17 dist-test-slave-x4qh krb5kdc[10861](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992597, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.96.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 05:03:17.743533 11098 init.cc:377] Logged in from keytab as kudu/127.9.96.124@KRBTEST.COM (short username kudu)
I20260806 05:03:17.744936 11098 webserver.cc:460] Webserver started at http://127.9.96.124:41489/ using document root <none> and password file <none>
I20260806 05:03:17.745559 11098 fs_manager.cc:359] Metadata directory not provided
I20260806 05:03:17.745644 11098 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 05:03:17.745904 11098 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 05:03:17.748762 11098 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785992227050689-9601-0/minicluster-data/master-2/data/instance:
uuid: "f6ad10e221f84c999f1cff2a89d4c179"
format_stamp: "Formatted at 2026-08-06 05:03:17 on dist-test-slave-x4qh"
server_key: "dba09f5a2042517db9551f8e4bf1dfd6"
server_key_iv: "3c541da1738b054a24d8f153a8ea2c17"
server_key_version: "encryptionkey@0"
I20260806 05:03:17.749480 11098 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785992227050689-9601-0/minicluster-data/master-2/wal/instance:
uuid: "f6ad10e221f84c999f1cff2a89d4c179"
format_stamp: "Formatted at 2026-08-06 05:03:17 on dist-test-slave-x4qh"
server_key: "dba09f5a2042517db9551f8e4bf1dfd6"
server_key_iv: "3c541da1738b054a24d8f153a8ea2c17"
server_key_version: "encryptionkey@0"
I20260806 05:03:17.754498 11098 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.004s
I20260806 05:03:17.759270 11125 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 05:03:17.760993 11098 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260806 05:03:17.761184 11098 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785992227050689-9601-0/minicluster-data/master-2/data,/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785992227050689-9601-0/minicluster-data/master-2/wal
uuid: "f6ad10e221f84c999f1cff2a89d4c179"
format_stamp: "Formatted at 2026-08-06 05:03:17 on dist-test-slave-x4qh"
server_key: "dba09f5a2042517db9551f8e4bf1dfd6"
server_key_iv: "3c541da1738b054a24d8f153a8ea2c17"
server_key_version: "encryptionkey@0"
I20260806 05:03:17.761451 11098 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785992227050689-9601-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785992227050689-9601-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785992227050689-9601-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 05:03:17.779183 11098 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 05:03:17.784319 11098 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 05:03:17.784654 11098 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 05:03:17.808104 11098 rpc_server.cc:307] RPC server started. Bound to: 127.9.96.124:39387
I20260806 05:03:17.808130 11197 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.96.124:39387 every 8 connection(s)
I20260806 05:03:17.809460 11098 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785992227050689-9601-0/minicluster-data/master-2/data/info.pb
I20260806 05:03:17.811476 9601 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu as pid 11098
I20260806 05:03:17.811669 9601 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785992227050689-9601-0/minicluster-data/master-2/wal/instance
I20260806 05:03:17.812263 9601 external_mini_cluster.cc:1428] Setting key f18ab5700a687b57937f35a461dbf5fc
I20260806 05:03:17.814383 11198 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 05:03:17.824648 11198 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.96.126" port: 43597 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 05:03:17 dist-test-slave-x4qh krb5kdc[10861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992596, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.96.124@KRBTEST.COM
Aug 06 05:03:17 dist-test-slave-x4qh krb5kdc[10861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992597, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.96.124@KRBTEST.COM for kudu/127.9.96.126@KRBTEST.COM
I20260806 05:03:17.837186 11202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:03:17.814980 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53938 (local address 127.9.96.124:39387)
0806 05:03:17.815841 (+ 861us) server_negotiation.cc:206] Beginning negotiation
0806 05:03:17.815857 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 05:03:17.815905 (+ 48us) server_negotiation.cc:407] Connection header received
0806 05:03:17.818355 (+ 2450us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 05:03:17.818388 (+ 33us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 05:03:17.818896 (+ 508us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 05:03:17.819351 (+ 455us) server_negotiation.cc:226] Negotiated authn=SASL
0806 05:03:17.820886 (+ 1535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:03:17.822497 (+ 1611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:03:17.823809 (+ 1312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:03:17.824502 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:03:17.824705 (+ 203us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:03:17.829962 (+ 5257us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 05:03:17.829969 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 05:03:17.829978 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 05:03:17.830010 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0806 05:03:17.833159 (+ 3149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 05:03:17.833816 (+ 657us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 05:03:17.833824 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 05:03:17.833832 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0806 05:03:17.834036 (+ 204us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 05:03:17.834498 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 05:03:17.834506 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 05:03:17.834510 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 05:03:17.835025 (+ 515us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 05:03:17.835265 (+ 240us) server_negotiation.cc:1035] Waiting for connection context
0806 05:03:17.835703 (+ 438us) server_negotiation.cc:299] Negotiation successful
0806 05:03:17.836106 (+ 403us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":642,"thread_start_us":430,"threads_started":1}
I20260806 05:03:17.843672 11205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:03:17.827319 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:43597 (local address 127.0.0.1:59742)
0806 05:03:17.828484 (+ 1165us) negotiation.cc:107] Waiting for socket to connect
0806 05:03:17.828520 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0806 05:03:17.828703 (+ 183us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 05:03:17.829135 (+ 432us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 05:03:17.829143 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 05:03:17.829978 (+ 835us) client_negotiation.cc:189] Negotiated authn=SASL
0806 05:03:17.830289 (+ 311us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 05:03:17.830299 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 05:03:17.831971 (+ 1672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 05:03:17.831975 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 05:03:17.833072 (+ 1097us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 05:03:17.833084 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 05:03:17.833863 (+ 779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 05:03:17.833868 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 05:03:17.834001 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:03:17.834626 (+ 625us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 05:03:17.834643 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 05:03:17.837138 (+ 2495us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 05:03:17.841083 (+ 3945us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 05:03:17.841093 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 05:03:17.841101 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0806 05:03:17.841686 (+ 585us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 05:03:17.842056 (+ 370us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 05:03:17.842061 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 05:03:17.842064 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 05:03:17.842231 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 05:03:17.842819 (+ 588us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 05:03:17.842824 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 05:03:17.842966 (+ 142us) client_negotiation.cc:769] Sending connection context
0806 05:03:17.843106 (+ 140us) client_negotiation.cc:240] Negotiation successful
0806 05:03:17.843309 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":977,"thread_start_us":851,"threads_started":1}
I20260806 05:03:17.844159 11204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:03:17.827427 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59742 (local address 127.9.96.126:43597)
0806 05:03:17.827991 (+ 564us) server_negotiation.cc:206] Beginning negotiation
0806 05:03:17.828004 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 05:03:17.828623 (+ 619us) server_negotiation.cc:407] Connection header received
0806 05:03:17.828864 (+ 241us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 05:03:17.828868 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 05:03:17.828945 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 05:03:17.829027 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0806 05:03:17.830450 (+ 1423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:03:17.831790 (+ 1340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:03:17.833232 (+ 1442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:03:17.833668 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:03:17.833814 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:03:17.837353 (+ 3539us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 05:03:17.837359 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 05:03:17.837364 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 05:03:17.837417 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0806 05:03:17.840942 (+ 3525us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 05:03:17.841814 (+ 872us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 05:03:17.841818 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 05:03:17.841820 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 05:03:17.841941 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 05:03:17.842354 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 05:03:17.842357 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 05:03:17.842359 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 05:03:17.842680 (+ 321us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 05:03:17.842896 (+ 216us) server_negotiation.cc:1035] Waiting for connection context
0806 05:03:17.843246 (+ 350us) server_negotiation.cc:299] Negotiation successful
0806 05:03:17.843501 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":423,"thread_start_us":269,"threads_started":1}
I20260806 05:03:17.845422 11198 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.96.125" port: 39223 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 05:03:17 dist-test-slave-x4qh krb5kdc[10861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992596, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.96.126@KRBTEST.COM
Aug 06 05:03:17 dist-test-slave-x4qh krb5kdc[10861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992597, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.96.124@KRBTEST.COM for kudu/127.9.96.125@KRBTEST.COM
I20260806 05:03:17.855513 11206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:03:17.840172 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59750 (local address 127.9.96.126:43597)
0806 05:03:17.840823 (+ 651us) server_negotiation.cc:206] Beginning negotiation
0806 05:03:17.840829 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 05:03:17.840855 (+ 26us) server_negotiation.cc:407] Connection header received
0806 05:03:17.840919 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 05:03:17.840923 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 05:03:17.840992 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 05:03:17.841092 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0806 05:03:17.842618 (+ 1526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:03:17.843855 (+ 1237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:03:17.845313 (+ 1458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:03:17.845832 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:03:17.845973 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:03:17.850002 (+ 4029us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 05:03:17.850008 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 05:03:17.850013 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 05:03:17.850048 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0806 05:03:17.852930 (+ 2882us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 05:03:17.853553 (+ 623us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 05:03:17.853558 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 05:03:17.853562 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 05:03:17.853681 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 05:03:17.854045 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 05:03:17.854050 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 05:03:17.854053 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 05:03:17.854324 (+ 271us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 05:03:17.854603 (+ 279us) server_negotiation.cc:1035] Waiting for connection context
0806 05:03:17.854898 (+ 295us) server_negotiation.cc:299] Negotiation successful
0806 05:03:17.855106 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":497,"thread_start_us":330,"threads_started":1}
I20260806 05:03:17.858433 11205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:03:17.846168 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.125:39223 (local address 127.0.0.1:60208)
0806 05:03:17.846325 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0806 05:03:17.846345 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0806 05:03:17.846479 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 05:03:17.846897 (+ 418us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 05:03:17.846901 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 05:03:17.847201 (+ 300us) client_negotiation.cc:189] Negotiated authn=SASL
0806 05:03:17.847331 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 05:03:17.847339 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 05:03:17.848318 (+ 979us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 05:03:17.848325 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 05:03:17.849269 (+ 944us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 05:03:17.849281 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 05:03:17.849997 (+ 716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 05:03:17.850001 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 05:03:17.850142 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:03:17.850665 (+ 523us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 05:03:17.850680 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0806 05:03:17.852523 (+ 1843us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 05:03:17.855863 (+ 3340us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 05:03:17.855870 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 05:03:17.855877 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0806 05:03:17.856278 (+ 401us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 05:03:17.856767 (+ 489us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 05:03:17.856774 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 05:03:17.856783 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0806 05:03:17.856928 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 05:03:17.857532 (+ 604us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 05:03:17.857537 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 05:03:17.857687 (+ 150us) client_negotiation.cc:769] Sending connection context
0806 05:03:17.857832 (+ 145us) client_negotiation.cc:240] Negotiation successful
0806 05:03:17.858089 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":55}
I20260806 05:03:17.858539 11091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:03:17.846165 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60208 (local address 127.9.96.125:39223)
0806 05:03:17.846348 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0806 05:03:17.846355 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 05:03:17.846410 (+ 55us) server_negotiation.cc:407] Connection header received
0806 05:03:17.846627 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 05:03:17.846633 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 05:03:17.846742 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 05:03:17.846863 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0806 05:03:17.847516 (+ 653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:03:17.848175 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:03:17.849422 (+ 1247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:03:17.849826 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:03:17.849986 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:03:17.852721 (+ 2735us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 05:03:17.852729 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 05:03:17.852734 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 05:03:17.852777 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0806 05:03:17.855743 (+ 2966us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 05:03:17.856449 (+ 706us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 05:03:17.856456 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 05:03:17.856460 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 05:03:17.856624 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 05:03:17.857051 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 05:03:17.857056 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 05:03:17.857059 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 05:03:17.857430 (+ 371us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 05:03:17.857573 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0806 05:03:17.857941 (+ 368us) server_negotiation.cc:299] Negotiation successful
0806 05:03:17.858170 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260806 05:03:17.859448 11198 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.96.124" port: 39387 } has no permanent_uuid. Determining permanent_uuid...
I20260806 05:03:17.859557 10983 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.96.124" port: 39387 } attempt: 4
Aug 06 05:03:17 dist-test-slave-x4qh krb5kdc[10861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992596, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.96.125@KRBTEST.COM
Aug 06 05:03:17 dist-test-slave-x4qh krb5kdc[10861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992596, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.96.126@KRBTEST.COM for kudu/127.9.96.124@KRBTEST.COM
Aug 06 05:03:17 dist-test-slave-x4qh krb5kdc[10861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992597, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.96.124@KRBTEST.COM for kudu/127.9.96.124@KRBTEST.COM
I20260806 05:03:17.875612 11202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:03:17.860236 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53946 (local address 127.9.96.124:39387)
0806 05:03:17.860967 (+ 731us) server_negotiation.cc:206] Beginning negotiation
0806 05:03:17.860975 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 05:03:17.860995 (+ 20us) server_negotiation.cc:407] Connection header received
0806 05:03:17.861051 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 05:03:17.861055 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 05:03:17.861132 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 05:03:17.861231 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0806 05:03:17.862066 (+ 835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:03:17.863021 (+ 955us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:03:17.864606 (+ 1585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:03:17.865066 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:03:17.865199 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:03:17.869062 (+ 3863us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 05:03:17.869068 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 05:03:17.869072 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 05:03:17.869112 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0806 05:03:17.872798 (+ 3686us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 05:03:17.873459 (+ 661us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 05:03:17.873465 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 05:03:17.873468 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 05:03:17.873660 (+ 192us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 05:03:17.874114 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 05:03:17.874126 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 05:03:17.874132 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 05:03:17.874448 (+ 316us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 05:03:17.874616 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0806 05:03:17.875005 (+ 389us) server_negotiation.cc:299] Negotiation successful
0806 05:03:17.875228 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":628}
I20260806 05:03:17.875829 11209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:03:17.857116 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60212 (local address 127.9.96.125:39223)
0806 05:03:17.857722 (+ 606us) server_negotiation.cc:206] Beginning negotiation
0806 05:03:17.857726 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 05:03:17.857744 (+ 18us) server_negotiation.cc:407] Connection header received
0806 05:03:17.857780 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 05:03:17.857790 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 05:03:17.857836 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 05:03:17.857951 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0806 05:03:17.859921 (+ 1970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:03:17.861064 (+ 1143us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:03:17.862825 (+ 1761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:03:17.863404 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:03:17.863531 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:03:17.869051 (+ 5520us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 05:03:17.869056 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 05:03:17.869059 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 05:03:17.869088 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0806 05:03:17.872798 (+ 3710us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 05:03:17.873614 (+ 816us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 05:03:17.873620 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 05:03:17.873623 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 05:03:17.873776 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 05:03:17.874194 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 05:03:17.874200 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 05:03:17.874206 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 05:03:17.874574 (+ 368us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 05:03:17.874827 (+ 253us) server_negotiation.cc:1035] Waiting for connection context
0806 05:03:17.875137 (+ 310us) server_negotiation.cc:299] Negotiation successful
0806 05:03:17.875415 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":519,"thread_start_us":318,"threads_started":1}
I20260806 05:03:17.875643 10985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:03:17.860162 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.124:39387 (local address 127.0.0.1:53946)
0806 05:03:17.860334 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0806 05:03:17.860352 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 05:03:17.860469 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 05:03:17.861263 (+ 794us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 05:03:17.861268 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 05:03:17.861727 (+ 459us) client_negotiation.cc:189] Negotiated authn=SASL
0806 05:03:17.861927 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 05:03:17.861940 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 05:03:17.863166 (+ 1226us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 05:03:17.863173 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 05:03:17.864450 (+ 1277us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 05:03:17.864464 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 05:03:17.865212 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 05:03:17.865218 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 05:03:17.865496 (+ 278us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:03:17.866047 (+ 551us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 05:03:17.866063 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 05:03:17.868840 (+ 2777us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 05:03:17.872978 (+ 4138us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 05:03:17.872981 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 05:03:17.872984 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 05:03:17.873325 (+ 341us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 05:03:17.873798 (+ 473us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 05:03:17.873806 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 05:03:17.873810 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 05:03:17.873981 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 05:03:17.874596 (+ 615us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 05:03:17.874601 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 05:03:17.874758 (+ 157us) client_negotiation.cc:769] Sending connection context
0806 05:03:17.874908 (+ 150us) client_negotiation.cc:240] Negotiation successful
0806 05:03:17.875194 (+ 286us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":57}
I20260806 05:03:17.879207 11205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:03:17.860505 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.124:39387 (local address 127.0.0.1:53956)
0806 05:03:17.860638 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0806 05:03:17.860651 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 05:03:17.860716 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 05:03:17.862477 (+ 1761us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 05:03:17.862482 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 05:03:17.862949 (+ 467us) client_negotiation.cc:189] Negotiated authn=SASL
0806 05:03:17.863133 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 05:03:17.863144 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 05:03:17.864613 (+ 1469us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 05:03:17.864618 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 05:03:17.866115 (+ 1497us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 05:03:17.866132 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 05:03:17.867214 (+ 1082us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 05:03:17.867223 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 05:03:17.867494 (+ 271us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:03:17.868038 (+ 544us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 05:03:17.868059 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0806 05:03:17.870716 (+ 2657us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 05:03:17.876213 (+ 5497us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 05:03:17.876220 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 05:03:17.876225 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 05:03:17.876742 (+ 517us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 05:03:17.877310 (+ 568us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 05:03:17.877315 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 05:03:17.877318 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 05:03:17.877476 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 05:03:17.878075 (+ 599us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 05:03:17.878079 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 05:03:17.878372 (+ 293us) client_negotiation.cc:769] Sending connection context
0806 05:03:17.878530 (+ 158us) client_negotiation.cc:240] Negotiation successful
0806 05:03:17.878837 (+ 307us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33}
I20260806 05:03:17.879968 11211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:03:17.860683 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53956 (local address 127.9.96.124:39387)
0806 05:03:17.862190 (+ 1507us) server_negotiation.cc:206] Beginning negotiation
0806 05:03:17.862196 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 05:03:17.862215 (+ 19us) server_negotiation.cc:407] Connection header received
0806 05:03:17.862258 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 05:03:17.862262 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 05:03:17.862321 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 05:03:17.862431 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0806 05:03:17.863257 (+ 826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:03:17.864473 (+ 1216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:03:17.866313 (+ 1840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:03:17.867015 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:03:17.867138 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:03:17.870941 (+ 3803us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 05:03:17.870945 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 05:03:17.870948 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 05:03:17.870977 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0806 05:03:17.874441 (+ 3464us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 05:03:17.876870 (+ 2429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 05:03:17.876875 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 05:03:17.876878 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 05:03:17.877034 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 05:03:17.877586 (+ 552us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 05:03:17.877593 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 05:03:17.877597 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 05:03:17.877933 (+ 336us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 05:03:17.878101 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0806 05:03:17.878677 (+ 576us) server_negotiation.cc:299] Negotiation successful
0806 05:03:17.878991 (+ 314us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":83,"server-negotiator.queue_time_us":492,"thread_start_us":611,"threads_started":1}
I20260806 05:03:17.881273 9601 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 05:03:17.881989 10983 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 800e308d265e4a9597974f5ca7ba342b: Bootstrap starting.
I20260806 05:03:17.885577 10983 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 800e308d265e4a9597974f5ca7ba342b: Neither blocks nor log segments found. Creating new log.
I20260806 05:03:17.886767 10983 log.cc:824] T 00000000000000000000000000000000 P 800e308d265e4a9597974f5ca7ba342b: Log is configured to *not* fsync() on all Append() calls
I20260806 05:03:17.884063 10947 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "800e308d265e4a9597974f5ca7ba342b"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59750
I20260806 05:03:17.888938 11198 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f6ad10e221f84c999f1cff2a89d4c179: Bootstrap starting.
I20260806 05:03:17.889879 10983 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 800e308d265e4a9597974f5ca7ba342b: No bootstrap required, opened a new log
I20260806 05:03:17.892740 11198 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f6ad10e221f84c999f1cff2a89d4c179: Neither blocks nor log segments found. Creating new log.
I20260806 05:03:17.893989 11198 log.cc:824] T 00000000000000000000000000000000 P f6ad10e221f84c999f1cff2a89d4c179: Log is configured to *not* fsync() on all Append() calls
I20260806 05:03:17.895495 10983 raft_consensus.cc:348] T 00000000000000000000000000000000 P 800e308d265e4a9597974f5ca7ba342b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "800e308d265e4a9597974f5ca7ba342b" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 43597 } } peers { permanent_uuid: "6e123e02193a4b0cab2fd9734c2cdda1" member_type: VOTER last_known_addr { host: "127.9.96.125" port: 39223 } } peers { permanent_uuid: "f6ad10e221f84c999f1cff2a89d4c179" member_type: VOTER last_known_addr { host: "127.9.96.124" port: 39387 } }
I20260806 05:03:17.895835 10983 raft_consensus.cc:729] T 00000000000000000000000000000000 P 800e308d265e4a9597974f5ca7ba342b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 800e308d265e4a9597974f5ca7ba342b, State: Initialized, Role: FOLLOWER
I20260806 05:03:17.896252 10983 consensus_queue.cc:260] T 00000000000000000000000000000000 P 800e308d265e4a9597974f5ca7ba342b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "800e308d265e4a9597974f5ca7ba342b" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 43597 } } peers { permanent_uuid: "6e123e02193a4b0cab2fd9734c2cdda1" member_type: VOTER last_known_addr { host: "127.9.96.125" port: 39223 } } peers { permanent_uuid: "f6ad10e221f84c999f1cff2a89d4c179" member_type: VOTER last_known_addr { host: "127.9.96.124" port: 39387 } }
I20260806 05:03:17.897300 11198 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f6ad10e221f84c999f1cff2a89d4c179: No bootstrap required, opened a new log
I20260806 05:03:17.897133 11216 sys_catalog.cc:455] T 00000000000000000000000000000000 P 800e308d265e4a9597974f5ca7ba342b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "800e308d265e4a9597974f5ca7ba342b" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 43597 } } peers { permanent_uuid: "6e123e02193a4b0cab2fd9734c2cdda1" member_type: VOTER last_known_addr { host: "127.9.96.125" port: 39223 } } peers { permanent_uuid: "f6ad10e221f84c999f1cff2a89d4c179" member_type: VOTER last_known_addr { host: "127.9.96.124" port: 39387 } } }
I20260806 05:03:17.897549 11216 sys_catalog.cc:458] T 00000000000000000000000000000000 P 800e308d265e4a9597974f5ca7ba342b [sys.catalog]: This master's current role is: FOLLOWER
I20260806 05:03:17.898262 10983 sys_catalog.cc:564] T 00000000000000000000000000000000 P 800e308d265e4a9597974f5ca7ba342b [sys.catalog]: configured and running, proceeding with master startup.
I20260806 05:03:17.898452 10947 raft_consensus.cc:482] T 00000000000000000000000000000000 P 800e308d265e4a9597974f5ca7ba342b [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260806 05:03:17.898672 10947 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 800e308d265e4a9597974f5ca7ba342b [term 0 FOLLOWER]: Advancing to term 1
I20260806 05:03:17.902230 10947 raft_consensus.cc:504] T 00000000000000000000000000000000 P 800e308d265e4a9597974f5ca7ba342b [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "800e308d265e4a9597974f5ca7ba342b" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 43597 } } peers { permanent_uuid: "6e123e02193a4b0cab2fd9734c2cdda1" member_type: VOTER last_known_addr { host: "127.9.96.125" port: 39223 } } peers { permanent_uuid: "f6ad10e221f84c999f1cff2a89d4c179" member_type: VOTER last_known_addr { host: "127.9.96.124" port: 39387 } }
I20260806 05:03:17.905786 10947 leader_election.cc:288] T 00000000000000000000000000000000 P 800e308d265e4a9597974f5ca7ba342b [CANDIDATE]: Term 1 election: Requested vote from peers 6e123e02193a4b0cab2fd9734c2cdda1 (127.9.96.125:39223), f6ad10e221f84c999f1cff2a89d4c179 (127.9.96.124:39387)
I20260806 05:03:17.908104 11198 raft_consensus.cc:348] T 00000000000000000000000000000000 P f6ad10e221f84c999f1cff2a89d4c179 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "800e308d265e4a9597974f5ca7ba342b" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 43597 } } peers { permanent_uuid: "6e123e02193a4b0cab2fd9734c2cdda1" member_type: VOTER last_known_addr { host: "127.9.96.125" port: 39223 } } peers { permanent_uuid: "f6ad10e221f84c999f1cff2a89d4c179" member_type: VOTER last_known_addr { host: "127.9.96.124" port: 39387 } }
I20260806 05:03:17.908592 11198 raft_consensus.cc:729] T 00000000000000000000000000000000 P f6ad10e221f84c999f1cff2a89d4c179 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f6ad10e221f84c999f1cff2a89d4c179, State: Initialized, Role: FOLLOWER
I20260806 05:03:17.909109 11198 consensus_queue.cc:260] T 00000000000000000000000000000000 P f6ad10e221f84c999f1cff2a89d4c179 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "800e308d265e4a9597974f5ca7ba342b" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 43597 } } peers { permanent_uuid: "6e123e02193a4b0cab2fd9734c2cdda1" member_type: VOTER last_known_addr { host: "127.9.96.125" port: 39223 } } peers { permanent_uuid: "f6ad10e221f84c999f1cff2a89d4c179" member_type: VOTER last_known_addr { host: "127.9.96.124" port: 39387 } }
I20260806 05:03:17.910578 11229 sys_catalog.cc:455] T 00000000000000000000000000000000 P f6ad10e221f84c999f1cff2a89d4c179 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "800e308d265e4a9597974f5ca7ba342b" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 43597 } } peers { permanent_uuid: "6e123e02193a4b0cab2fd9734c2cdda1" member_type: VOTER last_known_addr { host: "127.9.96.125" port: 39223 } } peers { permanent_uuid: "f6ad10e221f84c999f1cff2a89d4c179" member_type: VOTER last_known_addr { host: "127.9.96.124" port: 39387 } } }
I20260806 05:03:17.911643 11229 sys_catalog.cc:458] T 00000000000000000000000000000000 P f6ad10e221f84c999f1cff2a89d4c179 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 05:03:17.911731 11198 sys_catalog.cc:564] T 00000000000000000000000000000000 P f6ad10e221f84c999f1cff2a89d4c179 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 05:03:17.911791 11167 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "800e308d265e4a9597974f5ca7ba342b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "f6ad10e221f84c999f1cff2a89d4c179"
I20260806 05:03:17.912532 11167 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f6ad10e221f84c999f1cff2a89d4c179 [term 0 FOLLOWER]: Advancing to term 1
I20260806 05:03:17.918076 11167 raft_consensus.cc:2454] T 00000000000000000000000000000000 P f6ad10e221f84c999f1cff2a89d4c179 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 800e308d265e4a9597974f5ca7ba342b in term 1.
I20260806 05:03:17.918910 10905 leader_election.cc:302] T 00000000000000000000000000000000 P 800e308d265e4a9597974f5ca7ba342b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 800e308d265e4a9597974f5ca7ba342b, f6ad10e221f84c999f1cff2a89d4c179; no voters:
I20260806 05:03:17.919481 11216 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 800e308d265e4a9597974f5ca7ba342b [term 1 FOLLOWER]: Leader election won for term 1
I20260806 05:03:17.929050 11216 raft_consensus.cc:686] T 00000000000000000000000000000000 P 800e308d265e4a9597974f5ca7ba342b [term 1 LEADER]: Becoming Leader. State: Replica: 800e308d265e4a9597974f5ca7ba342b, State: Running, Role: LEADER
I20260806 05:03:17.930974 11216 consensus_queue.cc:237] T 00000000000000000000000000000000 P 800e308d265e4a9597974f5ca7ba342b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "800e308d265e4a9597974f5ca7ba342b" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 43597 } } peers { permanent_uuid: "6e123e02193a4b0cab2fd9734c2cdda1" member_type: VOTER last_known_addr { host: "127.9.96.125" port: 39223 } } peers { permanent_uuid: "f6ad10e221f84c999f1cff2a89d4c179" member_type: VOTER last_known_addr { host: "127.9.96.124" port: 39387 } }
I20260806 05:03:17.946265 11246 sys_catalog.cc:455] T 00000000000000000000000000000000 P 800e308d265e4a9597974f5ca7ba342b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 800e308d265e4a9597974f5ca7ba342b. Latest consensus state: current_term: 1 leader_uuid: "800e308d265e4a9597974f5ca7ba342b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "800e308d265e4a9597974f5ca7ba342b" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 43597 } } peers { permanent_uuid: "6e123e02193a4b0cab2fd9734c2cdda1" member_type: VOTER last_known_addr { host: "127.9.96.125" port: 39223 } } peers { permanent_uuid: "f6ad10e221f84c999f1cff2a89d4c179" member_type: VOTER last_known_addr { host: "127.9.96.124" port: 39387 } } }
I20260806 05:03:17.946853 11246 sys_catalog.cc:458] T 00000000000000000000000000000000 P 800e308d265e4a9597974f5ca7ba342b [sys.catalog]: This master's current role is: LEADER
I20260806 05:03:17.959463 11206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:03:17.916429 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59766 (local address 127.9.96.126:43597)
0806 05:03:17.916683 (+ 254us) server_negotiation.cc:206] Beginning negotiation
0806 05:03:17.916692 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 05:03:17.934920 (+ 18228us) server_negotiation.cc:407] Connection header received
0806 05:03:17.936057 (+ 1137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 05:03:17.936065 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 05:03:17.936168 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 05:03:17.936254 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0806 05:03:17.941571 (+ 5317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:03:17.943043 (+ 1472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:03:17.945907 (+ 2864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:03:17.946646 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:03:17.946767 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:03:17.951954 (+ 5187us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 05:03:17.951960 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 05:03:17.951965 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 05:03:17.952001 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0806 05:03:17.956045 (+ 4044us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 05:03:17.956947 (+ 902us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 05:03:17.956952 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 05:03:17.956956 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 05:03:17.957125 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 05:03:17.957592 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 05:03:17.957597 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 05:03:17.957600 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 05:03:17.957957 (+ 357us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 05:03:17.958219 (+ 262us) server_negotiation.cc:1035] Waiting for connection context
0806 05:03:17.958802 (+ 583us) server_negotiation.cc:299] Negotiation successful
0806 05:03:17.959050 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
W20260806 05:03:17.960944 10913 tablet.cc:2369] T 00000000000000000000000000000000 P 800e308d265e4a9597974f5ca7ba342b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 05:03:17.950261 10885 master_runner.cc:186] Error getting master registration for 127.9.96.125:39223: OK, instance_id { permanent_uuid: "6e123e02193a4b0cab2fd9734c2cdda1" instance_seqno: 1785992597272387 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260806 05:03:17.969501 10885 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260806 05:03:17.970101 11257 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 800e308d265e4a9597974f5ca7ba342b: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 05:03:17.970216 11257 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 05:03:17.970327 11259 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 05:03:17.972074 11209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:03:17.920067 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60214 (local address 127.9.96.125:39223)
0806 05:03:17.920296 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0806 05:03:17.920304 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 05:03:17.935529 (+ 15225us) server_negotiation.cc:407] Connection header received
0806 05:03:17.935844 (+ 315us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 05:03:17.935854 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 05:03:17.935976 (+ 122us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 05:03:17.936120 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0806 05:03:17.938927 (+ 2807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:03:17.940365 (+ 1438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:03:17.949395 (+ 9030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:03:17.950113 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:03:17.950247 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:03:17.964894 (+ 14647us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 05:03:17.964904 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 05:03:17.964909 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 05:03:17.964957 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0806 05:03:17.968768 (+ 3811us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 05:03:17.969599 (+ 831us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 05:03:17.969607 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 05:03:17.969612 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 05:03:17.969790 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 05:03:17.970263 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 05:03:17.970268 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 05:03:17.970271 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 05:03:17.970635 (+ 364us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 05:03:17.970885 (+ 250us) server_negotiation.cc:1035] Waiting for connection context
0806 05:03:17.971475 (+ 590us) server_negotiation.cc:299] Negotiation successful
0806 05:03:17.971725 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260806 05:03:17.972554 11259 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 05:03:17.986076 11167 raft_consensus.cc:1261] T 00000000000000000000000000000000 P f6ad10e221f84c999f1cff2a89d4c179 [term 1 FOLLOWER]: Refusing update from remote peer 800e308d265e4a9597974f5ca7ba342b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 05:03:17.987813 11246 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 800e308d265e4a9597974f5ca7ba342b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f6ad10e221f84c999f1cff2a89d4c179" member_type: VOTER last_known_addr { host: "127.9.96.124" port: 39387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 05:03:17.986896 11211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:03:17.935570 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53970 (local address 127.9.96.124:39387)
0806 05:03:17.935749 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0806 05:03:17.935755 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 05:03:17.944908 (+ 9153us) server_negotiation.cc:407] Connection header received
0806 05:03:17.948432 (+ 3524us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 05:03:17.948438 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 05:03:17.948525 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 05:03:17.948598 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0806 05:03:17.952884 (+ 4286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:03:17.954291 (+ 1407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:03:17.962738 (+ 8447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:03:17.963723 (+ 985us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:03:17.963853 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:03:17.978420 (+ 14567us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 05:03:17.978427 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 05:03:17.978432 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 05:03:17.978476 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0806 05:03:17.982713 (+ 4237us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 05:03:17.983666 (+ 953us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 05:03:17.983671 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 05:03:17.983674 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 05:03:17.983827 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 05:03:17.984341 (+ 514us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 05:03:17.984347 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 05:03:17.984350 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 05:03:17.985083 (+ 733us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 05:03:17.985497 (+ 414us) server_negotiation.cc:1035] Waiting for connection context
0806 05:03:17.985851 (+ 354us) server_negotiation.cc:299] Negotiation successful
0806 05:03:17.986245 (+ 394us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260806 05:03:17.996433 11083 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.96.124" port: 39387 } attempt: 5
Aug 06 05:03:18 dist-test-slave-x4qh krb5kdc[10861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992597, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.96.125@KRBTEST.COM for kudu/127.9.96.124@KRBTEST.COM
W20260806 05:03:18.024729 11271 catalog_manager.cc:1558] T 00000000000000000000000000000000 P f6ad10e221f84c999f1cff2a89d4c179: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 05:03:18.024865 11271 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 05:03:18.024650 11229 sys_catalog.cc:455] T 00000000000000000000000000000000 P f6ad10e221f84c999f1cff2a89d4c179 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 800e308d265e4a9597974f5ca7ba342b. Latest consensus state: current_term: 1 leader_uuid: "800e308d265e4a9597974f5ca7ba342b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "800e308d265e4a9597974f5ca7ba342b" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 43597 } } peers { permanent_uuid: "6e123e02193a4b0cab2fd9734c2cdda1" member_type: VOTER last_known_addr { host: "127.9.96.125" port: 39223 } } peers { permanent_uuid: "f6ad10e221f84c999f1cff2a89d4c179" member_type: VOTER last_known_addr { host: "127.9.96.124" port: 39387 } } }
I20260806 05:03:18.025025 11229 sys_catalog.cc:458] T 00000000000000000000000000000000 P f6ad10e221f84c999f1cff2a89d4c179 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 05:03:17.993544 11098 master_runner.cc:186] Error getting master registration for 127.9.96.125:39223: OK, instance_id { permanent_uuid: "6e123e02193a4b0cab2fd9734c2cdda1" instance_seqno: 1785992597272387 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260806 05:03:18.031177 11098 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260806 05:03:18.034426 11088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:03:17.997379 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.124:39387 (local address 127.0.0.1:53978)
0806 05:03:17.997685 (+ 306us) negotiation.cc:107] Waiting for socket to connect
0806 05:03:17.997708 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0806 05:03:17.997840 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 05:03:18.000981 (+ 3141us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 05:03:18.000986 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 05:03:18.001686 (+ 700us) client_negotiation.cc:189] Negotiated authn=SASL
0806 05:03:18.002004 (+ 318us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 05:03:18.002021 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 05:03:18.004101 (+ 2080us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 05:03:18.004110 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 05:03:18.006062 (+ 1952us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 05:03:18.006085 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 05:03:18.017618 (+ 11533us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 05:03:18.017628 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 05:03:18.017977 (+ 349us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:03:18.018804 (+ 827us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 05:03:18.018830 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0806 05:03:18.021875 (+ 3045us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 05:03:18.030964 (+ 9089us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 05:03:18.030970 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 05:03:18.030975 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 05:03:18.031604 (+ 629us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 05:03:18.032126 (+ 522us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 05:03:18.032144 (+ 18us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 05:03:18.032148 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 05:03:18.032370 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 05:03:18.033270 (+ 900us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 05:03:18.033276 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 05:03:18.033447 (+ 171us) client_negotiation.cc:769] Sending connection context
0806 05:03:18.033630 (+ 183us) client_negotiation.cc:240] Negotiation successful
0806 05:03:18.033947 (+ 317us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":154}
I20260806 05:03:18.036316 11211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:03:17.999796 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53978 (local address 127.9.96.124:39387)
0806 05:03:18.000425 (+ 629us) server_negotiation.cc:206] Beginning negotiation
0806 05:03:18.000463 (+ 38us) server_negotiation.cc:399] Waiting for connection header
0806 05:03:18.000482 (+ 19us) server_negotiation.cc:407] Connection header received
0806 05:03:18.000558 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 05:03:18.000563 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 05:03:18.000743 (+ 180us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 05:03:18.000973 (+ 230us) server_negotiation.cc:226] Negotiated authn=SASL
0806 05:03:18.002228 (+ 1255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:03:18.003895 (+ 1667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:03:18.006316 (+ 2421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:03:18.016973 (+ 10657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:03:18.017148 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:03:18.022135 (+ 4987us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 05:03:18.022142 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 05:03:18.022147 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 05:03:18.022186 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0806 05:03:18.030799 (+ 8613us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 05:03:18.031775 (+ 976us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 05:03:18.031781 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 05:03:18.031786 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 05:03:18.031971 (+ 185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 05:03:18.032544 (+ 573us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 05:03:18.032550 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 05:03:18.032554 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 05:03:18.032972 (+ 418us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 05:03:18.033175 (+ 203us) server_negotiation.cc:1035] Waiting for connection context
0806 05:03:18.035673 (+ 2498us) server_negotiation.cc:299] Negotiation successful
0806 05:03:18.035900 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":381}
W20260806 05:03:18.037111 11134 tablet.cc:2369] T 00000000000000000000000000000000 P f6ad10e221f84c999f1cff2a89d4c179: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 05:03:18.042452 11083 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6e123e02193a4b0cab2fd9734c2cdda1: Bootstrap starting.
I20260806 05:03:18.046890 11267 mvcc.cc:205] Tried to move back new op lower bound from 7315425681334292480 to 7315425681132843008. Current Snapshot: MvccSnapshot[applied={T|T < 7315425681334292480}]
I20260806 05:03:18.047706 11216 sys_catalog.cc:455] T 00000000000000000000000000000000 P 800e308d265e4a9597974f5ca7ba342b [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "800e308d265e4a9597974f5ca7ba342b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "800e308d265e4a9597974f5ca7ba342b" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 43597 } } peers { permanent_uuid: "6e123e02193a4b0cab2fd9734c2cdda1" member_type: VOTER last_known_addr { host: "127.9.96.125" port: 39223 } } peers { permanent_uuid: "f6ad10e221f84c999f1cff2a89d4c179" member_type: VOTER last_known_addr { host: "127.9.96.124" port: 39387 } } }
I20260806 05:03:18.048069 11216 sys_catalog.cc:458] T 00000000000000000000000000000000 P 800e308d265e4a9597974f5ca7ba342b [sys.catalog]: This master's current role is: LEADER
I20260806 05:03:18.049576 11083 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6e123e02193a4b0cab2fd9734c2cdda1: Neither blocks nor log segments found. Creating new log.
I20260806 05:03:18.054070 11083 log.cc:824] T 00000000000000000000000000000000 P 6e123e02193a4b0cab2fd9734c2cdda1: Log is configured to *not* fsync() on all Append() calls
I20260806 05:03:18.055835 11229 sys_catalog.cc:455] T 00000000000000000000000000000000 P f6ad10e221f84c999f1cff2a89d4c179 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "800e308d265e4a9597974f5ca7ba342b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "800e308d265e4a9597974f5ca7ba342b" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 43597 } } peers { permanent_uuid: "6e123e02193a4b0cab2fd9734c2cdda1" member_type: VOTER last_known_addr { host: "127.9.96.125" port: 39223 } } peers { permanent_uuid: "f6ad10e221f84c999f1cff2a89d4c179" member_type: VOTER last_known_addr { host: "127.9.96.124" port: 39387 } } }
I20260806 05:03:18.056146 11229 sys_catalog.cc:458] T 00000000000000000000000000000000 P f6ad10e221f84c999f1cff2a89d4c179 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 05:03:18.056180 11259 catalog_manager.cc:1347] Generated new cluster ID: c464eaefd55445ec8dcabd7b3759ad50
I20260806 05:03:18.056273 11259 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 05:03:18.063737 11083 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6e123e02193a4b0cab2fd9734c2cdda1: No bootstrap required, opened a new log
I20260806 05:03:18.078691 11259 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 05:03:18.081367 11083 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6e123e02193a4b0cab2fd9734c2cdda1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "800e308d265e4a9597974f5ca7ba342b" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 43597 } } peers { permanent_uuid: "6e123e02193a4b0cab2fd9734c2cdda1" member_type: VOTER last_known_addr { host: "127.9.96.125" port: 39223 } } peers { permanent_uuid: "f6ad10e221f84c999f1cff2a89d4c179" member_type: VOTER last_known_addr { host: "127.9.96.124" port: 39387 } }
I20260806 05:03:18.081857 11083 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6e123e02193a4b0cab2fd9734c2cdda1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e123e02193a4b0cab2fd9734c2cdda1, State: Initialized, Role: FOLLOWER
I20260806 05:03:18.082432 11083 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6e123e02193a4b0cab2fd9734c2cdda1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "800e308d265e4a9597974f5ca7ba342b" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 43597 } } peers { permanent_uuid: "6e123e02193a4b0cab2fd9734c2cdda1" member_type: VOTER last_known_addr { host: "127.9.96.125" port: 39223 } } peers { permanent_uuid: "f6ad10e221f84c999f1cff2a89d4c179" member_type: VOTER last_known_addr { host: "127.9.96.124" port: 39387 } }
I20260806 05:03:18.083106 11259 catalog_manager.cc:1504] Loading token signing keys...
I20260806 05:03:18.097728 11083 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6e123e02193a4b0cab2fd9734c2cdda1 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 05:03:18.098091 11053 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6e123e02193a4b0cab2fd9734c2cdda1 [term 0 FOLLOWER]: Advancing to term 1
I20260806 05:03:18.099475 11054 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "800e308d265e4a9597974f5ca7ba342b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "6e123e02193a4b0cab2fd9734c2cdda1"
I20260806 05:03:18.104252 11053 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 6e123e02193a4b0cab2fd9734c2cdda1 [term 1 FOLLOWER]: Refusing update from remote peer 800e308d265e4a9597974f5ca7ba342b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 05:03:18.107816 11054 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 6e123e02193a4b0cab2fd9734c2cdda1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 800e308d265e4a9597974f5ca7ba342b in term 1.
I20260806 05:03:18.110183 11246 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 800e308d265e4a9597974f5ca7ba342b [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e123e02193a4b0cab2fd9734c2cdda1" member_type: VOTER last_known_addr { host: "127.9.96.125" port: 39223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 05:03:18.097482 11280 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6e123e02193a4b0cab2fd9734c2cdda1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "800e308d265e4a9597974f5ca7ba342b" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 43597 } } peers { permanent_uuid: "6e123e02193a4b0cab2fd9734c2cdda1" member_type: VOTER last_known_addr { host: "127.9.96.125" port: 39223 } } peers { permanent_uuid: "f6ad10e221f84c999f1cff2a89d4c179" member_type: VOTER last_known_addr { host: "127.9.96.124" port: 39387 } } }
I20260806 05:03:18.117682 11259 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 800e308d265e4a9597974f5ca7ba342b: Generated new TSK 0
I20260806 05:03:18.118950 11259 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 05:03:18.119155 11280 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6e123e02193a4b0cab2fd9734c2cdda1 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 05:03:18.139041 11289 mvcc.cc:205] Tried to move back new op lower bound from 7315425681334292480 to 7315425681132843008. Current Snapshot: MvccSnapshot[applied={T|T < 7315425681334292480}]
I20260806 05:03:18.149138 11286 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6e123e02193a4b0cab2fd9734c2cdda1 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "800e308d265e4a9597974f5ca7ba342b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "800e308d265e4a9597974f5ca7ba342b" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 43597 } } peers { permanent_uuid: "6e123e02193a4b0cab2fd9734c2cdda1" member_type: VOTER last_known_addr { host: "127.9.96.125" port: 39223 } } peers { permanent_uuid: "f6ad10e221f84c999f1cff2a89d4c179" member_type: VOTER last_known_addr { host: "127.9.96.124" port: 39387 } } }
I20260806 05:03:18.149565 11286 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6e123e02193a4b0cab2fd9734c2cdda1 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 05:03:18.149878 11280 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6e123e02193a4b0cab2fd9734c2cdda1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 800e308d265e4a9597974f5ca7ba342b. Latest consensus state: current_term: 1 leader_uuid: "800e308d265e4a9597974f5ca7ba342b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "800e308d265e4a9597974f5ca7ba342b" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 43597 } } peers { permanent_uuid: "6e123e02193a4b0cab2fd9734c2cdda1" member_type: VOTER last_known_addr { host: "127.9.96.125" port: 39223 } } peers { permanent_uuid: "f6ad10e221f84c999f1cff2a89d4c179" member_type: VOTER last_known_addr { host: "127.9.96.124" port: 39387 } } }
I20260806 05:03:18.150121 11280 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6e123e02193a4b0cab2fd9734c2cdda1 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 05:03:18.198318 11312 catalog_manager.cc:1259] Loaded cluster ID: c464eaefd55445ec8dcabd7b3759ad50
I20260806 05:03:18.198483 11312 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 6e123e02193a4b0cab2fd9734c2cdda1: loading cluster ID for follower catalog manager: success
I20260806 05:03:18.202047 11312 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 6e123e02193a4b0cab2fd9734c2cdda1: acquiring CA information for follower catalog manager: success
I20260806 05:03:18.204437 11312 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 6e123e02193a4b0cab2fd9734c2cdda1: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260806 05:03:18.274046 11206 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 05:03:18.251108 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59778 (local address 127.9.96.126:43597)
0806 05:03:18.251334 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0806 05:03:18.251341 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 05:03:18.269701 (+ 18360us) server_negotiation.cc:407] Connection header received
0806 05:03:18.272311 (+ 2610us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 05:03:18.272318 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 05:03:18.272407 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 05:03:18.272481 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0806 05:03:18.273748 (+ 1267us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59778: BlockingRecv error: recv got EOF from 127.0.0.1:59778 (error 108)
Metrics: {"server-negotiator.queue_time_us":95}
W20260806 05:03:18.278461 11209 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 05:03:18.252883 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60216 (local address 127.9.96.125:39223)
0806 05:03:18.253070 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0806 05:03:18.253077 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 05:03:18.275926 (+ 22849us) server_negotiation.cc:407] Connection header received
0806 05:03:18.276696 (+ 770us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 05:03:18.276703 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 05:03:18.276805 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 05:03:18.276878 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0806 05:03:18.278231 (+ 1353us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60216: BlockingRecv error: recv got EOF from 127.0.0.1:60216 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
W20260806 05:03:18.280140 11211 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 05:03:18.270272 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53990 (local address 127.9.96.124:39387)
0806 05:03:18.270454 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0806 05:03:18.270459 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 05:03:18.276414 (+ 5955us) server_negotiation.cc:407] Connection header received
0806 05:03:18.278856 (+ 2442us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 05:03:18.278862 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 05:03:18.278944 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 05:03:18.279021 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0806 05:03:18.279913 (+ 892us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53990: BlockingRecv error: recv got EOF from 127.0.0.1:53990 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
I20260806 05:03:18.331619 11209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:03:18.312605 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60226 (local address 127.9.96.125:39223)
0806 05:03:18.312795 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0806 05:03:18.312802 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 05:03:18.323405 (+ 10603us) server_negotiation.cc:407] Connection header received
0806 05:03:18.323949 (+ 544us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 05:03:18.323956 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 05:03:18.324205 (+ 249us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 05:03:18.324358 (+ 153us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 05:03:18.325140 (+ 782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:03:18.326402 (+ 1262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:03:18.328493 (+ 2091us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:03:18.329216 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:03:18.329313 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:03:18.330047 (+ 734us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 05:03:18.330769 (+ 722us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 05:03:18.330888 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0806 05:03:18.331144 (+ 256us) server_negotiation.cc:299] Negotiation successful
0806 05:03:18.331257 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
W20260806 05:03:18.377995 11211 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 05:03:18.364853 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53998 (local address 127.9.96.124:39387)
0806 05:03:18.365062 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0806 05:03:18.365068 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 05:03:18.376022 (+ 10954us) server_negotiation.cc:407] Connection header received
0806 05:03:18.376451 (+ 429us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 05:03:18.376460 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 05:03:18.376585 (+ 125us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 05:03:18.376649 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0806 05:03:18.377759 (+ 1110us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53998: BlockingRecv error: recv got EOF from 127.0.0.1:53998 (error 108)
Metrics: {"server-negotiator.queue_time_us":73}
I20260806 05:03:18.437630 11209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:03:18.419824 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60242 (local address 127.9.96.125:39223)
0806 05:03:18.420011 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0806 05:03:18.420018 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 05:03:18.430596 (+ 10578us) server_negotiation.cc:407] Connection header received
0806 05:03:18.431106 (+ 510us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 05:03:18.431116 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 05:03:18.431230 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 05:03:18.431299 (+ 69us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 05:03:18.432008 (+ 709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:03:18.433165 (+ 1157us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:03:18.435156 (+ 1991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:03:18.435920 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:03:18.436027 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:03:18.436732 (+ 705us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 05:03:18.436891 (+ 159us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 05:03:18.436954 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0806 05:03:18.437197 (+ 243us) server_negotiation.cc:299] Negotiation successful
0806 05:03:18.437287 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
W20260806 05:03:18.487238 11211 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 05:03:18.474304 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54004 (local address 127.9.96.124:39387)
0806 05:03:18.474505 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0806 05:03:18.474511 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 05:03:18.485100 (+ 10589us) server_negotiation.cc:407] Connection header received
0806 05:03:18.485626 (+ 526us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 05:03:18.485632 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 05:03:18.485730 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 05:03:18.485811 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0806 05:03:18.486986 (+ 1175us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54004: BlockingRecv error: recv got EOF from 127.0.0.1:54004 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
I20260806 05:03:18.542992 11209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:03:18.524272 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60244 (local address 127.9.96.125:39223)
0806 05:03:18.524508 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0806 05:03:18.524515 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 05:03:18.535512 (+ 10997us) server_negotiation.cc:407] Connection header received
0806 05:03:18.536008 (+ 496us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 05:03:18.536014 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 05:03:18.536111 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 05:03:18.536204 (+ 93us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 05:03:18.536925 (+ 721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:03:18.538105 (+ 1180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:03:18.540524 (+ 2419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:03:18.541250 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:03:18.541355 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:03:18.542040 (+ 685us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 05:03:18.542235 (+ 195us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 05:03:18.542305 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0806 05:03:18.542562 (+ 257us) server_negotiation.cc:299] Negotiation successful
0806 05:03:18.542683 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
W20260806 05:03:18.592667 11211 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 05:03:18.579522 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54008 (local address 127.9.96.124:39387)
0806 05:03:18.579714 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0806 05:03:18.579724 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 05:03:18.590575 (+ 10851us) server_negotiation.cc:407] Connection header received
0806 05:03:18.591018 (+ 443us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 05:03:18.591024 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 05:03:18.591148 (+ 124us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 05:03:18.591261 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0806 05:03:18.592404 (+ 1143us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54008: BlockingRecv error: recv got EOF from 127.0.0.1:54008 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
I20260806 05:03:18.628819 11209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:03:18.621373 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60260 (local address 127.9.96.125:39223)
0806 05:03:18.621599 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0806 05:03:18.621607 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 05:03:18.622379 (+ 772us) server_negotiation.cc:407] Connection header received
0806 05:03:18.622586 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 05:03:18.622592 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 05:03:18.622701 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 05:03:18.622835 (+ 134us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 05:03:18.623348 (+ 513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:03:18.624589 (+ 1241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:03:18.626544 (+ 1955us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:03:18.627218 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:03:18.627350 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:03:18.627906 (+ 556us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 05:03:18.628092 (+ 186us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 05:03:18.628197 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0806 05:03:18.628421 (+ 224us) server_negotiation.cc:299] Negotiation successful
0806 05:03:18.628511 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":108}
W20260806 05:03:18.638571 11211 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 05:03:18.636098 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54018 (local address 127.9.96.124:39387)
0806 05:03:18.636432 (+ 334us) server_negotiation.cc:206] Beginning negotiation
0806 05:03:18.636443 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 05:03:18.636983 (+ 540us) server_negotiation.cc:407] Connection header received
0806 05:03:18.637175 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 05:03:18.637181 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 05:03:18.637289 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 05:03:18.637413 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0806 05:03:18.638339 (+ 926us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54018: BlockingRecv error: recv got EOF from 127.0.0.1:54018 (error 108)
Metrics: {"server-negotiator.queue_time_us":161}
I20260806 05:03:18.657692 11209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:03:18.650234 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60268 (local address 127.9.96.125:39223)
0806 05:03:18.650658 (+ 424us) server_negotiation.cc:206] Beginning negotiation
0806 05:03:18.650666 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 05:03:18.651274 (+ 608us) server_negotiation.cc:407] Connection header received
0806 05:03:18.651507 (+ 233us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 05:03:18.651513 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 05:03:18.651668 (+ 155us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 05:03:18.651772 (+ 104us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 05:03:18.652330 (+ 558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:03:18.653508 (+ 1178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:03:18.655494 (+ 1986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:03:18.656150 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:03:18.656278 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:03:18.656791 (+ 513us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 05:03:18.656958 (+ 167us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 05:03:18.657058 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0806 05:03:18.657234 (+ 176us) server_negotiation.cc:299] Negotiation successful
0806 05:03:18.657332 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":293}
W20260806 05:03:18.673669 11211 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 05:03:18.668047 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54032 (local address 127.9.96.124:39387)
0806 05:03:18.668233 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0806 05:03:18.668239 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 05:03:18.672079 (+ 3840us) server_negotiation.cc:407] Connection header received
0806 05:03:18.672261 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 05:03:18.672267 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 05:03:18.672355 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 05:03:18.672460 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0806 05:03:18.673385 (+ 925us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54032: BlockingRecv error: recv got EOF from 127.0.0.1:54032 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
I20260806 05:03:18.684347 11246 sys_catalog.cc:455] T 00000000000000000000000000000000 P 800e308d265e4a9597974f5ca7ba342b [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "800e308d265e4a9597974f5ca7ba342b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "800e308d265e4a9597974f5ca7ba342b" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 43597 } } peers { permanent_uuid: "6e123e02193a4b0cab2fd9734c2cdda1" member_type: VOTER last_known_addr { host: "127.9.96.125" port: 39223 } } peers { permanent_uuid: "f6ad10e221f84c999f1cff2a89d4c179" member_type: VOTER last_known_addr { host: "127.9.96.124" port: 39387 } } }
I20260806 05:03:18.684672 11246 sys_catalog.cc:458] T 00000000000000000000000000000000 P 800e308d265e4a9597974f5ca7ba342b [sys.catalog]: This master's current role is: LEADER
I20260806 05:03:18.706285 11209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:03:18.699549 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60274 (local address 127.9.96.125:39223)
0806 05:03:18.699704 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0806 05:03:18.699710 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 05:03:18.699878 (+ 168us) server_negotiation.cc:407] Connection header received
0806 05:03:18.700151 (+ 273us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 05:03:18.700156 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 05:03:18.700242 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 05:03:18.700335 (+ 93us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 05:03:18.700790 (+ 455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:03:18.701915 (+ 1125us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:03:18.703971 (+ 2056us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:03:18.704691 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:03:18.704852 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:03:18.705430 (+ 578us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 05:03:18.705609 (+ 179us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 05:03:18.705722 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0806 05:03:18.705885 (+ 163us) server_negotiation.cc:299] Negotiation successful
0806 05:03:18.705971 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
W20260806 05:03:18.719465 11211 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 05:03:18.717302 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54042 (local address 127.9.96.124:39387)
0806 05:03:18.717502 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0806 05:03:18.717508 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 05:03:18.718063 (+ 555us) server_negotiation.cc:407] Connection header received
0806 05:03:18.718239 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 05:03:18.718243 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 05:03:18.718313 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 05:03:18.718438 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0806 05:03:18.719207 (+ 769us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54042: BlockingRecv error: recv got EOF from 127.0.0.1:54042 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
I20260806 05:03:18.759467 11209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:03:18.754543 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60290 (local address 127.9.96.125:39223)
0806 05:03:18.754768 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0806 05:03:18.754775 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 05:03:18.754983 (+ 208us) server_negotiation.cc:407] Connection header received
0806 05:03:18.755137 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 05:03:18.755140 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 05:03:18.755207 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 05:03:18.755296 (+ 89us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 05:03:18.755702 (+ 406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:03:18.756517 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:03:18.757920 (+ 1403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:03:18.758336 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:03:18.758436 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:03:18.758781 (+ 345us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 05:03:18.758876 (+ 95us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 05:03:18.758940 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0806 05:03:18.759072 (+ 132us) server_negotiation.cc:299] Negotiation successful
0806 05:03:18.759131 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
W20260806 05:03:18.766460 11211 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 05:03:18.764644 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54050 (local address 127.9.96.124:39387)
0806 05:03:18.764871 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0806 05:03:18.764878 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 05:03:18.765085 (+ 207us) server_negotiation.cc:407] Connection header received
0806 05:03:18.765297 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 05:03:18.765301 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 05:03:18.765385 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 05:03:18.765465 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0806 05:03:18.766232 (+ 767us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54050: BlockingRecv error: recv got EOF from 127.0.0.1:54050 (error 108)
Metrics: {"server-negotiator.queue_time_us":82}
I20260806 05:03:18.838840 11209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:03:18.834025 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60306 (local address 127.9.96.125:39223)
0806 05:03:18.834279 (+ 254us) server_negotiation.cc:206] Beginning negotiation
0806 05:03:18.834286 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 05:03:18.834478 (+ 192us) server_negotiation.cc:407] Connection header received
0806 05:03:18.834637 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 05:03:18.834641 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 05:03:18.834709 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 05:03:18.834789 (+ 80us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 05:03:18.835184 (+ 395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:03:18.835985 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:03:18.837269 (+ 1284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:03:18.837686 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:03:18.837779 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:03:18.838122 (+ 343us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 05:03:18.838233 (+ 111us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 05:03:18.838300 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0806 05:03:18.838430 (+ 130us) server_negotiation.cc:299] Negotiation successful
0806 05:03:18.838492 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89}
W20260806 05:03:18.846266 11211 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 05:03:18.844531 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54052 (local address 127.9.96.124:39387)
0806 05:03:18.844696 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0806 05:03:18.844701 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 05:03:18.845159 (+ 458us) server_negotiation.cc:407] Connection header received
0806 05:03:18.845295 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 05:03:18.845298 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 05:03:18.845365 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 05:03:18.845444 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0806 05:03:18.846100 (+ 656us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54052: BlockingRecv error: recv got EOF from 127.0.0.1:54052 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
I20260806 05:03:18.983098 11209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:03:18.977983 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60316 (local address 127.9.96.125:39223)
0806 05:03:18.978266 (+ 283us) server_negotiation.cc:206] Beginning negotiation
0806 05:03:18.978272 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 05:03:18.978506 (+ 234us) server_negotiation.cc:407] Connection header received
0806 05:03:18.978662 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 05:03:18.978666 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 05:03:18.978735 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 05:03:18.978821 (+ 86us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 05:03:18.979240 (+ 419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:03:18.979950 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:03:18.981264 (+ 1314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:03:18.981696 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:03:18.981791 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:03:18.982244 (+ 453us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 05:03:18.982362 (+ 118us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 05:03:18.982453 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0806 05:03:18.982663 (+ 210us) server_negotiation.cc:299] Negotiation successful
0806 05:03:18.982768 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":122}
W20260806 05:03:18.991221 11211 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 05:03:18.989341 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54066 (local address 127.9.96.124:39387)
0806 05:03:18.989647 (+ 306us) server_negotiation.cc:206] Beginning negotiation
0806 05:03:18.989653 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 05:03:18.989822 (+ 169us) server_negotiation.cc:407] Connection header received
0806 05:03:18.990030 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 05:03:18.990037 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 05:03:18.990116 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 05:03:18.990226 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0806 05:03:18.991028 (+ 802us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54066: BlockingRecv error: recv got EOF from 127.0.0.1:54066 (error 108)
Metrics: {"server-negotiator.queue_time_us":150}
I20260806 05:03:19.026765 11271 catalog_manager.cc:1259] Loaded cluster ID: c464eaefd55445ec8dcabd7b3759ad50
I20260806 05:03:19.026865 11271 catalog_manager.cc:1552] T 00000000000000000000000000000000 P f6ad10e221f84c999f1cff2a89d4c179: loading cluster ID for follower catalog manager: success
I20260806 05:03:19.029327 11271 catalog_manager.cc:1574] T 00000000000000000000000000000000 P f6ad10e221f84c999f1cff2a89d4c179: acquiring CA information for follower catalog manager: success
I20260806 05:03:19.030952 11271 catalog_manager.cc:1602] T 00000000000000000000000000000000 P f6ad10e221f84c999f1cff2a89d4c179: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 05:03:19.257606 11209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:03:19.252277 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60332 (local address 127.9.96.125:39223)
0806 05:03:19.252551 (+ 274us) server_negotiation.cc:206] Beginning negotiation
0806 05:03:19.252561 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 05:03:19.252814 (+ 253us) server_negotiation.cc:407] Connection header received
0806 05:03:19.253055 (+ 241us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 05:03:19.253060 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 05:03:19.253129 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 05:03:19.253232 (+ 103us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 05:03:19.253729 (+ 497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:03:19.254436 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:03:19.255848 (+ 1412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:03:19.256277 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:03:19.256431 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:03:19.256882 (+ 451us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 05:03:19.256983 (+ 101us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 05:03:19.257086 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0806 05:03:19.257243 (+ 157us) server_negotiation.cc:299] Negotiation successful
0806 05:03:19.257322 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260806 05:03:19.269771 11211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:03:19.263815 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54070 (local address 127.9.96.124:39387)
0806 05:03:19.264022 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0806 05:03:19.264030 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 05:03:19.264174 (+ 144us) server_negotiation.cc:407] Connection header received
0806 05:03:19.264371 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 05:03:19.264377 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 05:03:19.264571 (+ 194us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 05:03:19.264697 (+ 126us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 05:03:19.265116 (+ 419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:03:19.265871 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:03:19.267392 (+ 1521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:03:19.268163 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:03:19.268297 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:03:19.268746 (+ 449us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 05:03:19.269305 (+ 559us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 05:03:19.269426 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0806 05:03:19.269503 (+ 77us) server_negotiation.cc:299] Negotiation successful
0806 05:03:19.269570 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260806 05:03:19.285561 11487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:03:19.275664 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59788 (local address 127.9.96.126:43597)
0806 05:03:19.277151 (+ 1487us) server_negotiation.cc:206] Beginning negotiation
0806 05:03:19.277160 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 05:03:19.277182 (+ 22us) server_negotiation.cc:407] Connection header received
0806 05:03:19.277330 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 05:03:19.277337 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 05:03:19.277570 (+ 233us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 05:03:19.277779 (+ 209us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 05:03:19.278945 (+ 1166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:03:19.280378 (+ 1433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:03:19.283018 (+ 2640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:03:19.283699 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:03:19.283820 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:03:19.284516 (+ 696us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 05:03:19.284904 (+ 388us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 05:03:19.285013 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0806 05:03:19.285161 (+ 148us) server_negotiation.cc:299] Negotiation successful
0806 05:03:19.285243 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1313,"thread_start_us":1194,"threads_started":1}
I20260806 05:03:19.290545 9601 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu pid 10885
I20260806 05:03:19.290858 11209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:03:19.275718 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60346 (local address 127.9.96.125:39223)
0806 05:03:19.275902 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0806 05:03:19.275909 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 05:03:19.278587 (+ 2678us) server_negotiation.cc:407] Connection header received
0806 05:03:19.278907 (+ 320us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 05:03:19.278912 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 05:03:19.278987 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 05:03:19.279083 (+ 96us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 05:03:19.279809 (+ 726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:03:19.280832 (+ 1023us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:03:19.287628 (+ 6796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:03:19.288211 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:03:19.288375 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:03:19.290080 (+ 1705us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 05:03:19.290207 (+ 127us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 05:03:19.290316 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0806 05:03:19.290475 (+ 159us) server_negotiation.cc:299] Negotiation successful
0806 05:03:19.290575 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260806 05:03:19.291688 10976 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 05:03:19.292827 11211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:03:19.278641 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54086 (local address 127.9.96.124:39387)
0806 05:03:19.278781 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0806 05:03:19.278786 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 05:03:19.285635 (+ 6849us) server_negotiation.cc:407] Connection header received
0806 05:03:19.286596 (+ 961us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 05:03:19.286600 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 05:03:19.286691 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 05:03:19.286789 (+ 98us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 05:03:19.287573 (+ 784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:03:19.288647 (+ 1074us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:03:19.290442 (+ 1795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:03:19.291164 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:03:19.291279 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:03:19.291761 (+ 482us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 05:03:19.291917 (+ 156us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 05:03:19.292020 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0806 05:03:19.292352 (+ 332us) server_negotiation.cc:299] Negotiation successful
0806 05:03:19.292469 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260806 05:03:19.400893 9601 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu with pid 10885
I20260806 05:03:19.433884 9601 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu pid 10993
I20260806 05:03:19.435036 11078 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 05:03:19.545258 9601 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu with pid 10993
I20260806 05:03:19.573079 9601 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu pid 11098
I20260806 05:03:19.574235 11192 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 05:03:19.687907 9601 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu with pid 11098
2026-08-06T05:03:19Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (5034 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785992227050689-9601-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 05:03:19 dist-test-slave-x4qh krb5kdc[11495](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 05:03:19 dist-test-slave-x4qh krb5kdc[11495](info): set up 2 sockets
Aug 06 05:03:19 dist-test-slave-x4qh krb5kdc[11495](info): commencing operation
krb5kdc: starting...
W20260806 05:03:21.030871 9601 mini_kdc.cc:121] Time spent starting KDC: real 1.294s user 0.001s sys 0.043s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785992227050689-9601-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785992227050689-9601-0/krb5kdc/test-user.keytab.
Aug 06 05:03:21 dist-test-slave-x4qh krb5kdc[11495](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992601, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T05:03:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T05:03:21Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.96.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.96.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.96.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.126.keytab.
Entry for principal kudu/127.9.96.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.126.keytab.
I20260806 05:03:21.346489 9601 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
/tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785992227050689-9601-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785992227050689-9601-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785992227050689-9601-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785992227050689-9601-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.96.126:33315
--webserver_interface=127.9.96.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.126.keytab
--principal=kudu/127.9.96.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.96.84:34019
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.96.126:33315,127.9.96.125:34707,127.9.96.124:44057
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785992227050689-9601-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785992227050689-9601-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785992227050689-9601-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 05:03:21.693984 11514 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260806 05:03:21.694348 11514 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 05:03:21.694479 11514 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 05:03:21.694618 11514 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 05:03:21.701891 11514 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 05:03:21.702009 11514 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 05:03:21.702139 11514 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 05:03:21.702225 11514 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 05:03:21.702332 11514 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 05:03:21.712504 11514 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.96.84:34019
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785992227050689-9601-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785992227050689-9601-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.96.126:33315,127.9.96.125:34707,127.9.96.124:44057
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.126.keytab
--principal=kudu/127.9.96.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.96.126:33315
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785992227050689-9601-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.96.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785992227050689-9601-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 03:56:12 UTC on ccycloud.quasar-guijbm.root.comops.site
build id 4569618
ASAN enabled
I20260806 05:03:21.713851 11514 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 05:03:21.715190 11514 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 05:03:21.727144 11524 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 05:03:21.727422 11525 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 05:03:21.727334 11528 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 05:03:21.728442 11514 server_base.cc:1034] running on GCE node
I20260806 05:03:21.729254 11514 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 05:03:21.730599 11514 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 05:03:21.731834 11514 hybrid_clock.cc:648] HybridClock initialized: now 1785992601731760 us; error 37 us; skew 500 ppm
I20260806 05:03:21.732177 11514 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 05:03:21 dist-test-slave-x4qh krb5kdc[11495](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992601, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.96.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 05:03:21.735272 11514 init.cc:377] Logged in from keytab as kudu/127.9.96.126@KRBTEST.COM (short username kudu)
I20260806 05:03:21.736814 11514 webserver.cc:460] Webserver started at http://127.9.96.126:35297/ using document root <none> and password file <none>
I20260806 05:03:21.737495 11514 fs_manager.cc:359] Metadata directory not provided
I20260806 05:03:21.737632 11514 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 05:03:21.737977 11514 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 05:03:21.740978 11514 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785992227050689-9601-0/minicluster-data/master-0/data/instance:
uuid: "49d7212a4740491a803a558fc032823a"
format_stamp: "Formatted at 2026-08-06 05:03:21 on dist-test-slave-x4qh"
server_key: "c2124f063e966d2574da213eed6e256b"
server_key_iv: "8c5dc48bb047659554a28966dd275306"
server_key_version: "encryptionkey@0"
I20260806 05:03:21.741775 11514 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785992227050689-9601-0/minicluster-data/master-0/wal/instance:
uuid: "49d7212a4740491a803a558fc032823a"
format_stamp: "Formatted at 2026-08-06 05:03:21 on dist-test-slave-x4qh"
server_key: "c2124f063e966d2574da213eed6e256b"
server_key_iv: "8c5dc48bb047659554a28966dd275306"
server_key_version: "encryptionkey@0"
I20260806 05:03:21.746806 11514 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260806 05:03:21.750155 11540 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 05:03:21.751549 11514 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 05:03:21.751734 11514 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785992227050689-9601-0/minicluster-data/master-0/data,/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785992227050689-9601-0/minicluster-data/master-0/wal
uuid: "49d7212a4740491a803a558fc032823a"
format_stamp: "Formatted at 2026-08-06 05:03:21 on dist-test-slave-x4qh"
server_key: "c2124f063e966d2574da213eed6e256b"
server_key_iv: "8c5dc48bb047659554a28966dd275306"
server_key_version: "encryptionkey@0"
I20260806 05:03:21.751952 11514 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785992227050689-9601-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785992227050689-9601-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785992227050689-9601-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 05:03:21.769961 11514 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 05:03:21.774140 11514 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 05:03:21.774421 11514 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 05:03:21.793167 11514 rpc_server.cc:307] RPC server started. Bound to: 127.9.96.126:33315
I20260806 05:03:21.793201 11616 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.96.126:33315 every 8 connection(s)
I20260806 05:03:21.794468 11514 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785992227050689-9601-0/minicluster-data/master-0/data/info.pb
I20260806 05:03:21.797619 9601 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu as pid 11514
I20260806 05:03:21.797809 9601 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785992227050689-9601-0/minicluster-data/master-0/wal/instance
I20260806 05:03:21.798214 11618 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 05:03:21.798280 9601 external_mini_cluster.cc:1428] Setting key e838652c14bc470f5ef00b14c7440f41
I20260806 05:03:21.804802 11618 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.96.126" port: 33315 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 05:03:21 dist-test-slave-x4qh krb5kdc[11495](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992601, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.96.126@KRBTEST.COM for kudu/127.9.96.126@KRBTEST.COM
I20260806 05:03:21.831167 11623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:03:21.809440 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52410 (local address 127.9.96.126:33315)
0806 05:03:21.810240 (+ 800us) server_negotiation.cc:206] Beginning negotiation
0806 05:03:21.810246 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 05:03:21.810297 (+ 51us) server_negotiation.cc:407] Connection header received
0806 05:03:21.812558 (+ 2261us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 05:03:21.812570 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 05:03:21.812923 (+ 353us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 05:03:21.813156 (+ 233us) server_negotiation.cc:226] Negotiated authn=SASL
0806 05:03:21.814643 (+ 1487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:03:21.816861 (+ 2218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:03:21.818244 (+ 1383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:03:21.819441 (+ 1197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:03:21.819650 (+ 209us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:03:21.823709 (+ 4059us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 05:03:21.823718 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 05:03:21.823730 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 05:03:21.823764 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0806 05:03:21.826708 (+ 2944us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 05:03:21.827766 (+ 1058us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 05:03:21.827773 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 05:03:21.827775 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 05:03:21.827926 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 05:03:21.828436 (+ 510us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 05:03:21.828440 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 05:03:21.828442 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 05:03:21.828922 (+ 480us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 05:03:21.829086 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0806 05:03:21.829864 (+ 778us) server_negotiation.cc:299] Negotiation successful
0806 05:03:21.830113 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":608,"thread_start_us":439,"threads_started":1}
I20260806 05:03:21.831277 11622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:03:21.808292 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:33315 (local address 127.0.0.1:52410)
0806 05:03:21.809149 (+ 857us) negotiation.cc:107] Waiting for socket to connect
0806 05:03:21.809198 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0806 05:03:21.812139 (+ 2941us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 05:03:21.813189 (+ 1050us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 05:03:21.813194 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 05:03:21.814030 (+ 836us) client_negotiation.cc:189] Negotiated authn=SASL
0806 05:03:21.814410 (+ 380us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 05:03:21.814427 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 05:03:21.817025 (+ 2598us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 05:03:21.817029 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 05:03:21.818083 (+ 1054us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 05:03:21.818099 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 05:03:21.819657 (+ 1558us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 05:03:21.819662 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 05:03:21.819811 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:03:21.820506 (+ 695us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 05:03:21.820532 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0806 05:03:21.823481 (+ 2949us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 05:03:21.826862 (+ 3381us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 05:03:21.826879 (+ 17us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 05:03:21.826897 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0806 05:03:21.827617 (+ 720us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 05:03:21.828062 (+ 445us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 05:03:21.828071 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 05:03:21.828079 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0806 05:03:21.828315 (+ 236us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 05:03:21.829089 (+ 774us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 05:03:21.829101 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 05:03:21.829380 (+ 279us) client_negotiation.cc:769] Sending connection context
0806 05:03:21.829727 (+ 347us) client_negotiation.cc:240] Negotiation successful
0806 05:03:21.830312 (+ 585us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":541,"mutex_wait_us":1,"thread_start_us":316,"threads_started":1}
WARNING: no policy specified for kudu/127.9.96.125@KRBTEST.COM; defaulting to no policy
I20260806 05:03:21.834218 11618 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.96.125" port: 34707 } has no permanent_uuid. Determining permanent_uuid...
I20260806 05:03:21.835477 11622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:03:21.834954 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.125:34707 (local address 127.0.0.1:55670)
0806 05:03:21.835196 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0806 05:03:21.835303 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.96.125:34707: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":131}
W20260806 05:03:21.835788 11545 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.9.96.125:34707: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.96.125@KRBTEST.COM" created.
I20260806 05:03:21.837447 11622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:03:21.837022 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.125:34707 (local address 127.0.0.1:55686)
0806 05:03:21.837206 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0806 05:03:21.837311 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.96.125:34707: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260806 05:03:21.837664 11618 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.96.125:34707: Network error: Client connection negotiation failed: client connection to 127.9.96.125:34707: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.96.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.125.keytab.
Entry for principal kudu/127.9.96.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.125.keytab.
I20260806 05:03:21.874171 9601 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
/tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785992227050689-9601-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785992227050689-9601-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785992227050689-9601-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785992227050689-9601-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.96.125:34707
--webserver_interface=127.9.96.125
--webserver_port=0
--keytab_file=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.125.keytab
--principal=kudu/127.9.96.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.96.84:34019
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.96.126:33315,127.9.96.125:34707,127.9.96.124:44057
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785992227050689-9601-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785992227050689-9601-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785992227050689-9601-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 05:03:21.887022 11618 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.96.125" port: 34707 } attempt: 1
I20260806 05:03:21.888336 11622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:03:21.887731 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.125:34707 (local address 127.0.0.1:55690)
0806 05:03:21.887991 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0806 05:03:21.888126 (+ 135us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.96.125:34707: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
I20260806 05:03:21.889627 11622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:03:21.889100 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.125:34707 (local address 127.0.0.1:55700)
0806 05:03:21.889360 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0806 05:03:21.889456 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.96.125:34707: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
W20260806 05:03:21.889866 11618 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.96.125:34707: Network error: Client connection negotiation failed: client connection to 127.9.96.125:34707: connect: Connection refused (error 111)
I20260806 05:03:21.958158 11618 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.96.125" port: 34707 } attempt: 2
I20260806 05:03:21.959602 11622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:03:21.958915 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.125:34707 (local address 127.0.0.1:55714)
0806 05:03:21.959181 (+ 266us) negotiation.cc:107] Waiting for socket to connect
0806 05:03:21.959329 (+ 148us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.96.125:34707: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":87}
I20260806 05:03:21.960733 11622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:03:21.960276 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.125:34707 (local address 127.0.0.1:55720)
0806 05:03:21.960442 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0806 05:03:21.960552 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.96.125:34707: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260806 05:03:21.960975 11618 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.96.125:34707: Network error: Client connection negotiation failed: client connection to 127.9.96.125:34707: connect: Connection refused (error 111)
I20260806 05:03:22.052249 11618 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.96.125" port: 34707 } attempt: 3
I20260806 05:03:22.053542 11622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:03:22.053092 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.125:34707 (local address 127.0.0.1:55736)
0806 05:03:22.053302 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0806 05:03:22.053400 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.96.125:34707: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":83}
I20260806 05:03:22.054678 11622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:03:22.054336 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.125:34707 (local address 127.0.0.1:55738)
0806 05:03:22.054479 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0806 05:03:22.054547 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.96.125:34707: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
W20260806 05:03:22.054994 11618 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.96.125:34707: Network error: Client connection negotiation failed: client connection to 127.9.96.125:34707: connect: Connection refused (error 111)
I20260806 05:03:22.198277 11618 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.96.125" port: 34707 } attempt: 4
I20260806 05:03:22.199630 11622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:03:22.199003 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.125:34707 (local address 127.0.0.1:55752)
0806 05:03:22.199355 (+ 352us) negotiation.cc:107] Waiting for socket to connect
0806 05:03:22.199500 (+ 145us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.96.125:34707: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":173}
I20260806 05:03:22.200649 11622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:03:22.200269 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.125:34707 (local address 127.0.0.1:55754)
0806 05:03:22.200438 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0806 05:03:22.200517 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.96.125:34707: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260806 05:03:22.200902 11618 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.96.125:34707: Network error: Client connection negotiation failed: client connection to 127.9.96.125:34707: connect: Connection refused (error 111)
W20260806 05:03:22.217967 11628 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260806 05:03:22.218274 11628 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 05:03:22.218379 11628 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 05:03:22.218515 11628 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 05:03:22.225767 11628 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 05:03:22.225847 11628 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 05:03:22.225943 11628 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 05:03:22.226027 11628 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 05:03:22.226136 11628 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 05:03:22.236011 11628 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.96.84:34019
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785992227050689-9601-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785992227050689-9601-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.9.96.126:33315,127.9.96.125:34707,127.9.96.124:44057
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.125.keytab
--principal=kudu/127.9.96.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.96.125:34707
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785992227050689-9601-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.9.96.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785992227050689-9601-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 03:56:12 UTC on ccycloud.quasar-guijbm.root.comops.site
build id 4569618
ASAN enabled
I20260806 05:03:22.237313 11628 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 05:03:22.238581 11628 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 05:03:22.250445 11639 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 05:03:22.250550 11635 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 05:03:22.250524 11636 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 05:03:22.251235 11628 server_base.cc:1034] running on GCE node
I20260806 05:03:22.252048 11628 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 05:03:22.253402 11628 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 05:03:22.254649 11628 hybrid_clock.cc:648] HybridClock initialized: now 1785992602254589 us; error 51 us; skew 500 ppm
I20260806 05:03:22.255016 11628 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 05:03:22 dist-test-slave-x4qh krb5kdc[11495](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992602, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.96.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 05:03:22.258335 11628 init.cc:377] Logged in from keytab as kudu/127.9.96.125@KRBTEST.COM (short username kudu)
I20260806 05:03:22.259819 11628 webserver.cc:460] Webserver started at http://127.9.96.125:38509/ using document root <none> and password file <none>
I20260806 05:03:22.260509 11628 fs_manager.cc:359] Metadata directory not provided
I20260806 05:03:22.260650 11628 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 05:03:22.260974 11628 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 05:03:22.263962 11628 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785992227050689-9601-0/minicluster-data/master-1/data/instance:
uuid: "5a65793af729464ca850ffef626261fc"
format_stamp: "Formatted at 2026-08-06 05:03:22 on dist-test-slave-x4qh"
server_key: "fd1dba24f4c39816a5db4cb998753b34"
server_key_iv: "15245b54b83a4fc453e31c10ed0c022e"
server_key_version: "encryptionkey@0"
I20260806 05:03:22.264809 11628 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785992227050689-9601-0/minicluster-data/master-1/wal/instance:
uuid: "5a65793af729464ca850ffef626261fc"
format_stamp: "Formatted at 2026-08-06 05:03:22 on dist-test-slave-x4qh"
server_key: "fd1dba24f4c39816a5db4cb998753b34"
server_key_iv: "15245b54b83a4fc453e31c10ed0c022e"
server_key_version: "encryptionkey@0"
I20260806 05:03:22.269824 11628 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260806 05:03:22.273182 11648 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 05:03:22.274435 11628 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260806 05:03:22.274629 11628 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785992227050689-9601-0/minicluster-data/master-1/data,/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785992227050689-9601-0/minicluster-data/master-1/wal
uuid: "5a65793af729464ca850ffef626261fc"
format_stamp: "Formatted at 2026-08-06 05:03:22 on dist-test-slave-x4qh"
server_key: "fd1dba24f4c39816a5db4cb998753b34"
server_key_iv: "15245b54b83a4fc453e31c10ed0c022e"
server_key_version: "encryptionkey@0"
I20260806 05:03:22.274845 11628 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785992227050689-9601-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785992227050689-9601-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785992227050689-9601-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 05:03:22.296806 11628 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 05:03:22.300717 11628 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 05:03:22.301002 11628 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 05:03:22.320106 11628 rpc_server.cc:307] RPC server started. Bound to: 127.9.96.125:34707
I20260806 05:03:22.320124 11730 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.96.125:34707 every 8 connection(s)
I20260806 05:03:22.321488 11628 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785992227050689-9601-0/minicluster-data/master-1/data/info.pb
I20260806 05:03:22.325318 9601 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu as pid 11628
I20260806 05:03:22.325511 11732 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 05:03:22.325533 9601 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785992227050689-9601-0/minicluster-data/master-1/wal/instance
I20260806 05:03:22.326000 9601 external_mini_cluster.cc:1428] Setting key d737900edee9b23c8ff16693b25f111e
I20260806 05:03:22.331982 11732 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.96.126" port: 33315 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 05:03:22 dist-test-slave-x4qh krb5kdc[11495](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992602, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.96.125@KRBTEST.COM for kudu/127.9.96.126@KRBTEST.COM
I20260806 05:03:22.353235 11734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:03:22.334503 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52422 (local address 127.9.96.126:33315)
0806 05:03:22.335005 (+ 502us) server_negotiation.cc:206] Beginning negotiation
0806 05:03:22.335009 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 05:03:22.336006 (+ 997us) server_negotiation.cc:407] Connection header received
0806 05:03:22.337886 (+ 1880us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 05:03:22.337892 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 05:03:22.337986 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 05:03:22.338108 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0806 05:03:22.339731 (+ 1623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:03:22.340445 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:03:22.342057 (+ 1612us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:03:22.342819 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:03:22.342976 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:03:22.346965 (+ 3989us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 05:03:22.346970 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 05:03:22.346973 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 05:03:22.347017 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0806 05:03:22.349838 (+ 2821us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 05:03:22.350794 (+ 956us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 05:03:22.350798 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 05:03:22.350800 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 05:03:22.350916 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 05:03:22.351467 (+ 551us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 05:03:22.351471 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 05:03:22.351473 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 05:03:22.351740 (+ 267us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 05:03:22.351863 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0806 05:03:22.352812 (+ 949us) server_negotiation.cc:299] Negotiation successful
0806 05:03:22.352968 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":402,"thread_start_us":268,"threads_started":1}
I20260806 05:03:22.354772 11735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:03:22.335045 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:33315 (local address 127.0.0.1:52422)
0806 05:03:22.335823 (+ 778us) negotiation.cc:107] Waiting for socket to connect
0806 05:03:22.335868 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0806 05:03:22.337610 (+ 1742us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 05:03:22.338348 (+ 738us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 05:03:22.338356 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 05:03:22.338855 (+ 499us) client_negotiation.cc:189] Negotiated authn=SASL
0806 05:03:22.339495 (+ 640us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 05:03:22.339524 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 05:03:22.340621 (+ 1097us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 05:03:22.340627 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 05:03:22.341888 (+ 1261us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 05:03:22.341911 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 05:03:22.343033 (+ 1122us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 05:03:22.343037 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 05:03:22.343222 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:03:22.343955 (+ 733us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 05:03:22.343976 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0806 05:03:22.346736 (+ 2760us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 05:03:22.349984 (+ 3248us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 05:03:22.349996 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 05:03:22.350014 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0806 05:03:22.350669 (+ 655us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 05:03:22.351034 (+ 365us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 05:03:22.351041 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 05:03:22.351048 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0806 05:03:22.351273 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 05:03:22.351874 (+ 601us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 05:03:22.351885 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 05:03:22.352387 (+ 502us) client_negotiation.cc:769] Sending connection context
0806 05:03:22.352730 (+ 343us) client_negotiation.cc:240] Negotiation successful
0806 05:03:22.353258 (+ 528us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":547,"thread_start_us":348,"threads_started":1}
I20260806 05:03:22.356696 11732 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.96.125" port: 34707 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.9.96.124@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.96.124@KRBTEST.COM" created.
Aug 06 05:03:22 dist-test-slave-x4qh krb5kdc[11495](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992602, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.96.125@KRBTEST.COM for kudu/127.9.96.125@KRBTEST.COM
I20260806 05:03:22.370939 11735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:03:22.357468 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.125:34707 (local address 127.0.0.1:55758)
0806 05:03:22.357668 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0806 05:03:22.357686 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 05:03:22.357791 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 05:03:22.358628 (+ 837us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 05:03:22.358635 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 05:03:22.359093 (+ 458us) client_negotiation.cc:189] Negotiated authn=SASL
0806 05:03:22.359308 (+ 215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 05:03:22.359323 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 05:03:22.360770 (+ 1447us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 05:03:22.360774 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 05:03:22.361699 (+ 925us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 05:03:22.361711 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 05:03:22.362631 (+ 920us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 05:03:22.362634 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 05:03:22.362744 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:03:22.363305 (+ 561us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 05:03:22.363332 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0806 05:03:22.365534 (+ 2202us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 05:03:22.368549 (+ 3015us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 05:03:22.368556 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 05:03:22.368562 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0806 05:03:22.368913 (+ 351us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 05:03:22.369271 (+ 358us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 05:03:22.369278 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 05:03:22.369283 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 05:03:22.369428 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 05:03:22.369981 (+ 553us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 05:03:22.369985 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 05:03:22.370118 (+ 133us) client_negotiation.cc:769] Sending connection context
0806 05:03:22.370284 (+ 166us) client_negotiation.cc:240] Negotiation successful
0806 05:03:22.370552 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":70}
I20260806 05:03:22.371075 11738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:03:22.357592 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55758 (local address 127.9.96.125:34707)
0806 05:03:22.358222 (+ 630us) server_negotiation.cc:206] Beginning negotiation
0806 05:03:22.358228 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 05:03:22.358253 (+ 25us) server_negotiation.cc:407] Connection header received
0806 05:03:22.358298 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 05:03:22.358304 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 05:03:22.358456 (+ 152us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 05:03:22.358572 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0806 05:03:22.359527 (+ 955us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:03:22.360636 (+ 1109us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:03:22.361828 (+ 1192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:03:22.362509 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:03:22.362606 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:03:22.365737 (+ 3131us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 05:03:22.365749 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 05:03:22.365760 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 05:03:22.365787 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0806 05:03:22.368406 (+ 2619us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 05:03:22.369042 (+ 636us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 05:03:22.369058 (+ 16us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 05:03:22.369060 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 05:03:22.369163 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 05:03:22.369532 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 05:03:22.369539 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 05:03:22.369542 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 05:03:22.369858 (+ 316us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 05:03:22.370065 (+ 207us) server_negotiation.cc:1035] Waiting for connection context
0806 05:03:22.370494 (+ 429us) server_negotiation.cc:299] Negotiation successful
0806 05:03:22.370712 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":477,"thread_start_us":335,"threads_started":1}
I20260806 05:03:22.373553 11732 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.96.124" port: 44057 } has no permanent_uuid. Determining permanent_uuid...
I20260806 05:03:22.374756 11735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:03:22.374303 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.124:44057 (local address 127.0.0.1:47408)
0806 05:03:22.374484 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0806 05:03:22.374624 (+ 140us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.96.124:44057: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260806 05:03:22.375043 11652 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.9.96.124:44057: connect: Connection refused (error 111)
I20260806 05:03:22.376663 11735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:03:22.376345 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.124:44057 (local address 127.0.0.1:47424)
0806 05:03:22.376490 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0806 05:03:22.376565 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.96.124:44057: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
W20260806 05:03:22.376919 11732 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.96.124:44057: Network error: Client connection negotiation failed: client connection to 127.9.96.124:44057: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.96.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.124.keytab.
Entry for principal kudu/127.9.96.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.124.keytab.
I20260806 05:03:22.398972 9601 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
/tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785992227050689-9601-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785992227050689-9601-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785992227050689-9601-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785992227050689-9601-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.96.124:44057
--webserver_interface=127.9.96.124
--webserver_port=0
--keytab_file=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.124.keytab
--principal=kudu/127.9.96.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.96.84:34019
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.96.126:33315,127.9.96.125:34707,127.9.96.124:44057
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785992227050689-9601-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785992227050689-9601-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785992227050689-9601-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 05:03:22.426223 11732 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.96.124" port: 44057 } attempt: 1
I20260806 05:03:22.427606 11735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:03:22.426936 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.124:44057 (local address 127.0.0.1:47432)
0806 05:03:22.427204 (+ 268us) negotiation.cc:107] Waiting for socket to connect
0806 05:03:22.427366 (+ 162us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.96.124:44057: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":81}
I20260806 05:03:22.428757 11735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:03:22.428265 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.124:44057 (local address 127.0.0.1:47438)
0806 05:03:22.428476 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0806 05:03:22.428592 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.96.124:44057: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
W20260806 05:03:22.429003 11732 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.96.124:44057: Network error: Client connection negotiation failed: client connection to 127.9.96.124:44057: connect: Connection refused (error 111)
I20260806 05:03:22.497273 11732 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.96.124" port: 44057 } attempt: 2
I20260806 05:03:22.498574 11735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:03:22.498032 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.124:44057 (local address 127.0.0.1:47440)
0806 05:03:22.498260 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0806 05:03:22.498381 (+ 121us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.96.124:44057: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":81}
I20260806 05:03:22.499631 11735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:03:22.499212 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.124:44057 (local address 127.0.0.1:47454)
0806 05:03:22.499414 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0806 05:03:22.499497 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.96.124:44057: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260806 05:03:22.499876 11732 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.96.124:44057: Network error: Client connection negotiation failed: client connection to 127.9.96.124:44057: connect: Connection refused (error 111)
I20260806 05:03:22.500165 11618 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.96.125" port: 34707 } attempt: 5
Aug 06 05:03:22 dist-test-slave-x4qh krb5kdc[11495](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992601, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.96.126@KRBTEST.COM for kudu/127.9.96.125@KRBTEST.COM
I20260806 05:03:22.514453 11738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:03:22.500852 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55762 (local address 127.9.96.125:34707)
0806 05:03:22.501036 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0806 05:03:22.501040 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 05:03:22.501058 (+ 18us) server_negotiation.cc:407] Connection header received
0806 05:03:22.501256 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 05:03:22.501260 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 05:03:22.501337 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 05:03:22.501430 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0806 05:03:22.502559 (+ 1129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:03:22.503236 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:03:22.504975 (+ 1739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:03:22.505417 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:03:22.505513 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:03:22.509023 (+ 3510us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 05:03:22.509028 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 05:03:22.509032 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 05:03:22.509064 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0806 05:03:22.511860 (+ 2796us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 05:03:22.512507 (+ 647us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 05:03:22.512515 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 05:03:22.512518 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 05:03:22.512642 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 05:03:22.513044 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 05:03:22.513049 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 05:03:22.513053 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 05:03:22.513327 (+ 274us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 05:03:22.513494 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0806 05:03:22.513919 (+ 425us) server_negotiation.cc:299] Negotiation successful
0806 05:03:22.514153 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":98}
I20260806 05:03:22.514453 11622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:03:22.500771 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.125:34707 (local address 127.0.0.1:55762)
0806 05:03:22.500978 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0806 05:03:22.500998 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0806 05:03:22.501128 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 05:03:22.501501 (+ 373us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 05:03:22.501506 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 05:03:22.502099 (+ 593us) client_negotiation.cc:189] Negotiated authn=SASL
0806 05:03:22.502396 (+ 297us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 05:03:22.502413 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 05:03:22.503423 (+ 1010us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 05:03:22.503430 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 05:03:22.504818 (+ 1388us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 05:03:22.504834 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 05:03:22.505589 (+ 755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 05:03:22.505595 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 05:03:22.505799 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:03:22.506406 (+ 607us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 05:03:22.506429 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0806 05:03:22.508825 (+ 2396us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 05:03:22.511998 (+ 3173us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 05:03:22.512004 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 05:03:22.512009 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 05:03:22.512387 (+ 378us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 05:03:22.512763 (+ 376us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 05:03:22.512768 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 05:03:22.512772 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 05:03:22.512920 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 05:03:22.513472 (+ 552us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 05:03:22.513477 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 05:03:22.513631 (+ 154us) client_negotiation.cc:769] Sending connection context
0806 05:03:22.513805 (+ 174us) client_negotiation.cc:240] Negotiation successful
0806 05:03:22.514085 (+ 280us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":69}
I20260806 05:03:22.515545 11618 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.96.124" port: 44057 } has no permanent_uuid. Determining permanent_uuid...
I20260806 05:03:22.516685 11622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:03:22.516302 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.124:44057 (local address 127.0.0.1:47466)
0806 05:03:22.516469 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0806 05:03:22.516550 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.96.124:44057: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260806 05:03:22.517670 11622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:03:22.517344 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.124:44057 (local address 127.0.0.1:47478)
0806 05:03:22.517502 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0806 05:03:22.517577 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.96.124:44057: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260806 05:03:22.517946 11618 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.96.124:44057: Network error: Client connection negotiation failed: client connection to 127.9.96.124:44057: connect: Connection refused (error 111)
I20260806 05:03:22.569288 11618 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.96.124" port: 44057 } attempt: 1
I20260806 05:03:22.570525 11622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:03:22.570068 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.124:44057 (local address 127.0.0.1:47488)
0806 05:03:22.570269 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0806 05:03:22.570359 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.96.124:44057: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
I20260806 05:03:22.571607 11622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:03:22.571214 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.124:44057 (local address 127.0.0.1:47494)
0806 05:03:22.571423 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0806 05:03:22.571504 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.96.124:44057: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260806 05:03:22.571810 11618 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.96.124:44057: Network error: Client connection negotiation failed: client connection to 127.9.96.124:44057: connect: Connection refused (error 111)
I20260806 05:03:22.591116 11732 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.96.124" port: 44057 } attempt: 3
I20260806 05:03:22.592281 11735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:03:22.591800 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.124:44057 (local address 127.0.0.1:47496)
0806 05:03:22.592033 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0806 05:03:22.592123 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.96.124:44057: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":82}
I20260806 05:03:22.593364 11735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:03:22.592964 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.124:44057 (local address 127.0.0.1:47502)
0806 05:03:22.593176 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0806 05:03:22.593260 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.96.124:44057: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":134}
W20260806 05:03:22.593654 11732 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.96.124:44057: Network error: Client connection negotiation failed: client connection to 127.9.96.124:44057: connect: Connection refused (error 111)
I20260806 05:03:22.640041 11618 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.96.124" port: 44057 } attempt: 2
I20260806 05:03:22.641213 11622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:03:22.640743 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.124:44057 (local address 127.0.0.1:47516)
0806 05:03:22.640972 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0806 05:03:22.641055 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.96.124:44057: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
I20260806 05:03:22.642153 11622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:03:22.641826 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.124:44057 (local address 127.0.0.1:47528)
0806 05:03:22.641988 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0806 05:03:22.642058 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.96.124:44057: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
W20260806 05:03:22.642355 11618 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.96.124:44057: Network error: Client connection negotiation failed: client connection to 127.9.96.124:44057: connect: Connection refused (error 111)
I20260806 05:03:22.736878 11732 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.96.124" port: 44057 } attempt: 4
I20260806 05:03:22.738118 11735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:03:22.737593 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.124:44057 (local address 127.0.0.1:47542)
0806 05:03:22.737848 (+ 255us) negotiation.cc:107] Waiting for socket to connect
0806 05:03:22.737963 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.96.124:44057: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
I20260806 05:03:22.739079 11735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:03:22.738761 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.124:44057 (local address 127.0.0.1:47556)
0806 05:03:22.738906 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0806 05:03:22.738978 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.96.124:44057: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260806 05:03:22.739295 11732 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.96.124:44057: Network error: Client connection negotiation failed: client connection to 127.9.96.124:44057: connect: Connection refused (error 111)
W20260806 05:03:22.740309 11742 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260806 05:03:22.740664 11742 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 05:03:22.740809 11742 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 05:03:22.740943 11742 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 05:03:22.748188 11742 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 05:03:22.748306 11742 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 05:03:22.748431 11742 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 05:03:22.748520 11742 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 05:03:22.748628 11742 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 05:03:22.748574 11618 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.96.124" port: 44057 } attempt: 3
I20260806 05:03:22.749688 11622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:03:22.749180 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.124:44057 (local address 127.0.0.1:47572)
0806 05:03:22.749391 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0806 05:03:22.749493 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.96.124:44057: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
I20260806 05:03:22.750730 11622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:03:22.750310 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.124:44057 (local address 127.0.0.1:47576)
0806 05:03:22.750516 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0806 05:03:22.750603 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.96.124:44057: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":93}
W20260806 05:03:22.750953 11618 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.96.124:44057: Network error: Client connection negotiation failed: client connection to 127.9.96.124:44057: connect: Connection refused (error 111)
I20260806 05:03:22.759354 11742 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.96.84:34019
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785992227050689-9601-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785992227050689-9601-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.9.96.126:33315,127.9.96.125:34707,127.9.96.124:44057
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785992227050689-9601-0/krb5kdc/kudu_127.9.96.124.keytab
--principal=kudu/127.9.96.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.96.124:44057
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785992227050689-9601-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.9.96.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785992227050689-9601-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 03:56:12 UTC on ccycloud.quasar-guijbm.root.comops.site
build id 4569618
ASAN enabled
I20260806 05:03:22.760710 11742 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 05:03:22.761919 11742 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 05:03:22.773469 11751 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 05:03:22.773536 11748 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 05:03:22.773634 11749 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 05:03:22.774565 11742 server_base.cc:1034] running on GCE node
I20260806 05:03:22.775228 11742 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 05:03:22.776556 11742 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 05:03:22.777778 11742 hybrid_clock.cc:648] HybridClock initialized: now 1785992602777725 us; error 29 us; skew 500 ppm
I20260806 05:03:22.778065 11742 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 05:03:22 dist-test-slave-x4qh krb5kdc[11495](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992602, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.96.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 05:03:22.781262 11742 init.cc:377] Logged in from keytab as kudu/127.9.96.124@KRBTEST.COM (short username kudu)
I20260806 05:03:22.782577 11742 webserver.cc:460] Webserver started at http://127.9.96.124:34571/ using document root <none> and password file <none>
I20260806 05:03:22.783218 11742 fs_manager.cc:359] Metadata directory not provided
I20260806 05:03:22.783304 11742 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 05:03:22.783640 11742 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 05:03:22.786492 11742 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785992227050689-9601-0/minicluster-data/master-2/data/instance:
uuid: "f14c3b06acd44971bbd9af7750df7af4"
format_stamp: "Formatted at 2026-08-06 05:03:22 on dist-test-slave-x4qh"
server_key: "ddd01914f05f20821afa5246dafcf11e"
server_key_iv: "13cdde2c9bea45cb81188e29b37106f5"
server_key_version: "encryptionkey@0"
I20260806 05:03:22.787222 11742 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785992227050689-9601-0/minicluster-data/master-2/wal/instance:
uuid: "f14c3b06acd44971bbd9af7750df7af4"
format_stamp: "Formatted at 2026-08-06 05:03:22 on dist-test-slave-x4qh"
server_key: "ddd01914f05f20821afa5246dafcf11e"
server_key_iv: "13cdde2c9bea45cb81188e29b37106f5"
server_key_version: "encryptionkey@0"
I20260806 05:03:22.792351 11742 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260806 05:03:22.796579 11764 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 05:03:22.798099 11742 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260806 05:03:22.798266 11742 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785992227050689-9601-0/minicluster-data/master-2/data,/tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785992227050689-9601-0/minicluster-data/master-2/wal
uuid: "f14c3b06acd44971bbd9af7750df7af4"
format_stamp: "Formatted at 2026-08-06 05:03:22 on dist-test-slave-x4qh"
server_key: "ddd01914f05f20821afa5246dafcf11e"
server_key_iv: "13cdde2c9bea45cb81188e29b37106f5"
server_key_version: "encryptionkey@0"
I20260806 05:03:22.798478 11742 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785992227050689-9601-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785992227050689-9601-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785992227050689-9601-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 05:03:22.812404 11742 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 05:03:22.816655 11742 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 05:03:22.816951 11742 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 05:03:22.834944 11742 rpc_server.cc:307] RPC server started. Bound to: 127.9.96.124:44057
I20260806 05:03:22.834951 11836 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.96.124:44057 every 8 connection(s)
I20260806 05:03:22.836450 11742 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785992227050689-9601-0/minicluster-data/master-2/data/info.pb
I20260806 05:03:22.839709 9601 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu as pid 11742
I20260806 05:03:22.839910 9601 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOfb9uZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785992227050689-9601-0/minicluster-data/master-2/wal/instance
I20260806 05:03:22.840349 11837 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 05:03:22.840497 9601 external_mini_cluster.cc:1428] Setting key f7fa333eda750aa830d0786cf0d6db34
I20260806 05:03:22.850329 11837 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.96.126" port: 33315 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 05:03:22 dist-test-slave-x4qh krb5kdc[11495](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992601, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.96.124@KRBTEST.COM
Aug 06 05:03:22 dist-test-slave-x4qh krb5kdc[11495](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992602, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.96.124@KRBTEST.COM for kudu/127.9.96.126@KRBTEST.COM
I20260806 05:03:22.864320 11841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:03:22.842245 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47586 (local address 127.9.96.124:44057)
0806 05:03:22.843014 (+ 769us) server_negotiation.cc:206] Beginning negotiation
0806 05:03:22.843029 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 05:03:22.843076 (+ 47us) server_negotiation.cc:407] Connection header received
0806 05:03:22.845299 (+ 2223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 05:03:22.845328 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 05:03:22.845798 (+ 470us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 05:03:22.846215 (+ 417us) server_negotiation.cc:226] Negotiated authn=SASL
0806 05:03:22.847622 (+ 1407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:03:22.849593 (+ 1971us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:03:22.850828 (+ 1235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:03:22.851844 (+ 1016us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:03:22.852029 (+ 185us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:03:22.856893 (+ 4864us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 05:03:22.856900 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 05:03:22.856909 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 05:03:22.856943 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0806 05:03:22.860249 (+ 3306us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 05:03:22.861093 (+ 844us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 05:03:22.861101 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 05:03:22.861109 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0806 05:03:22.861308 (+ 199us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 05:03:22.861657 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 05:03:22.861664 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 05:03:22.861670 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 05:03:22.862363 (+ 693us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 05:03:22.862681 (+ 318us) server_negotiation.cc:1035] Waiting for connection context
0806 05:03:22.863024 (+ 343us) server_negotiation.cc:299] Negotiation successful
0806 05:03:22.863293 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":542,"thread_start_us":324,"threads_started":1}
I20260806 05:03:22.869213 11734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:03:22.853115 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52424 (local address 127.9.96.126:33315)
0806 05:03:22.853281 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0806 05:03:22.853293 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 05:03:22.853911 (+ 618us) server_negotiation.cc:407] Connection header received
0806 05:03:22.854115 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 05:03:22.854121 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 05:03:22.854213 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 05:03:22.854322 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0806 05:03:22.855658 (+ 1336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:03:22.856505 (+ 847us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:03:22.858140 (+ 1635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:03:22.858888 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:03:22.859035 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:03:22.862692 (+ 3657us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 05:03:22.862698 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 05:03:22.862704 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 05:03:22.862746 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0806 05:03:22.866064 (+ 3318us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 05:03:22.866888 (+ 824us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 05:03:22.866894 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 05:03:22.866897 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 05:03:22.867072 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 05:03:22.867543 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 05:03:22.867548 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 05:03:22.867551 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 05:03:22.867914 (+ 363us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 05:03:22.868065 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0806 05:03:22.868620 (+ 555us) server_negotiation.cc:299] Negotiation successful
0806 05:03:22.868877 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260806 05:03:22.869210 11842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:03:22.853144 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.126:33315 (local address 127.0.0.1:52424)
0806 05:03:22.853777 (+ 633us) negotiation.cc:107] Waiting for socket to connect
0806 05:03:22.853811 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0806 05:03:22.853978 (+ 167us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 05:03:22.854397 (+ 419us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 05:03:22.854404 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 05:03:22.855125 (+ 721us) client_negotiation.cc:189] Negotiated authn=SASL
0806 05:03:22.855499 (+ 374us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 05:03:22.855518 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 05:03:22.856658 (+ 1140us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 05:03:22.856663 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 05:03:22.857979 (+ 1316us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 05:03:22.857991 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 05:03:22.859078 (+ 1087us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 05:03:22.859083 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 05:03:22.859200 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:03:22.859829 (+ 629us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 05:03:22.859849 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0806 05:03:22.862487 (+ 2638us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 05:03:22.866205 (+ 3718us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 05:03:22.866223 (+ 18us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 05:03:22.866229 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0806 05:03:22.866751 (+ 522us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 05:03:22.867203 (+ 452us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 05:03:22.867208 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 05:03:22.867211 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 05:03:22.867413 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 05:03:22.868072 (+ 659us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 05:03:22.868083 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 05:03:22.868282 (+ 199us) client_negotiation.cc:769] Sending connection context
0806 05:03:22.868478 (+ 196us) client_negotiation.cc:240] Negotiation successful
0806 05:03:22.868801 (+ 323us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":450,"thread_start_us":344,"threads_started":1}
I20260806 05:03:22.870563 11837 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.96.125" port: 34707 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 05:03:22 dist-test-slave-x4qh krb5kdc[11495](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992601, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.96.126@KRBTEST.COM
Aug 06 05:03:22 dist-test-slave-x4qh krb5kdc[11495](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992602, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.96.124@KRBTEST.COM for kudu/127.9.96.125@KRBTEST.COM
I20260806 05:03:22.884405 11844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:03:22.867313 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52430 (local address 127.9.96.126:33315)
0806 05:03:22.868477 (+ 1164us) server_negotiation.cc:206] Beginning negotiation
0806 05:03:22.868481 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 05:03:22.868501 (+ 20us) server_negotiation.cc:407] Connection header received
0806 05:03:22.868540 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 05:03:22.868543 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 05:03:22.868598 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 05:03:22.868714 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0806 05:03:22.870277 (+ 1563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:03:22.871141 (+ 864us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:03:22.872985 (+ 1844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:03:22.873624 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:03:22.873748 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:03:22.878252 (+ 4504us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 05:03:22.878256 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 05:03:22.878260 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 05:03:22.878296 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0806 05:03:22.881336 (+ 3040us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 05:03:22.882094 (+ 758us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 05:03:22.882097 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 05:03:22.882100 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 05:03:22.882206 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 05:03:22.882642 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 05:03:22.882646 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 05:03:22.882649 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 05:03:22.883015 (+ 366us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 05:03:22.883286 (+ 271us) server_negotiation.cc:1035] Waiting for connection context
0806 05:03:22.883713 (+ 427us) server_negotiation.cc:299] Negotiation successful
0806 05:03:22.883942 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1040,"thread_start_us":812,"threads_started":1}
I20260806 05:03:22.885294 11842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:03:22.871336 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.125:34707 (local address 127.0.0.1:55778)
0806 05:03:22.871610 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0806 05:03:22.871627 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 05:03:22.871730 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 05:03:22.872050 (+ 320us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 05:03:22.872055 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 05:03:22.872475 (+ 420us) client_negotiation.cc:189] Negotiated authn=SASL
0806 05:03:22.872649 (+ 174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 05:03:22.872662 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 05:03:22.873982 (+ 1320us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 05:03:22.873988 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 05:03:22.875503 (+ 1515us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 05:03:22.875520 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 05:03:22.876236 (+ 716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 05:03:22.876242 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 05:03:22.876431 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:03:22.876983 (+ 552us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 05:03:22.877000 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 05:03:22.879151 (+ 2151us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 05:03:22.882542 (+ 3391us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 05:03:22.882546 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 05:03:22.882548 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 05:03:22.883047 (+ 499us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 05:03:22.883514 (+ 467us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 05:03:22.883518 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 05:03:22.883520 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 05:03:22.883711 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 05:03:22.884312 (+ 601us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 05:03:22.884320 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 05:03:22.884509 (+ 189us) client_negotiation.cc:769] Sending connection context
0806 05:03:22.884662 (+ 153us) client_negotiation.cc:240] Negotiation successful
0806 05:03:22.884946 (+ 284us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":175}
I20260806 05:03:22.885344 11738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:03:22.871318 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55778 (local address 127.9.96.125:34707)
0806 05:03:22.871525 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0806 05:03:22.871530 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 05:03:22.871695 (+ 165us) server_negotiation.cc:407] Connection header received
0806 05:03:22.871844 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 05:03:22.871848 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 05:03:22.871918 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 05:03:22.871994 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0806 05:03:22.872802 (+ 808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:03:22.873821 (+ 1019us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:03:22.875650 (+ 1829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:03:22.876073 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:03:22.876174 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:03:22.879341 (+ 3167us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 05:03:22.879346 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 05:03:22.879349 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 05:03:22.879434 (+ 85us) server_negotiation.cc:922] calling sasl_server_start()
0806 05:03:22.882424 (+ 2990us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 05:03:22.883161 (+ 737us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 05:03:22.883166 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 05:03:22.883169 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 05:03:22.883352 (+ 183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 05:03:22.883844 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 05:03:22.883851 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 05:03:22.883855 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 05:03:22.884172 (+ 317us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 05:03:22.884297 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0806 05:03:22.884759 (+ 462us) server_negotiation.cc:299] Negotiation successful
0806 05:03:22.884986 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90}
I20260806 05:03:22.886241 11837 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.96.124" port: 44057 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 05:03:22 dist-test-slave-x4qh krb5kdc[11495](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992602, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.96.124@KRBTEST.COM for kudu/127.9.96.124@KRBTEST.COM
Aug 06 05:03:22 dist-test-slave-x4qh krb5kdc[11495](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992601, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.96.125@KRBTEST.COM
I20260806 05:03:22.899711 11842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:03:22.887000 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.124:44057 (local address 127.0.0.1:47594)
0806 05:03:22.887145 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0806 05:03:22.887164 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0806 05:03:22.887256 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 05:03:22.887729 (+ 473us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 05:03:22.887732 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 05:03:22.888046 (+ 314us) client_negotiation.cc:189] Negotiated authn=SASL
0806 05:03:22.888163 (+ 117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 05:03:22.888170 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 05:03:22.889074 (+ 904us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 05:03:22.889081 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 05:03:22.889906 (+ 825us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 05:03:22.889916 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 05:03:22.890605 (+ 689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 05:03:22.890611 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 05:03:22.890817 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:03:22.891449 (+ 632us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 05:03:22.891474 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0806 05:03:22.893650 (+ 2176us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 05:03:22.897208 (+ 3558us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 05:03:22.897212 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 05:03:22.897214 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 05:03:22.897545 (+ 331us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 05:03:22.897924 (+ 379us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 05:03:22.897927 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 05:03:22.897930 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 05:03:22.898046 (+ 116us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 05:03:22.898664 (+ 618us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 05:03:22.898669 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 05:03:22.898846 (+ 177us) client_negotiation.cc:769] Sending connection context
0806 05:03:22.899036 (+ 190us) client_negotiation.cc:240] Negotiation successful
0806 05:03:22.899355 (+ 319us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":48}
I20260806 05:03:22.900033 11841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:03:22.887224 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47594 (local address 127.9.96.124:44057)
0806 05:03:22.887444 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0806 05:03:22.887451 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 05:03:22.887471 (+ 20us) server_negotiation.cc:407] Connection header received
0806 05:03:22.887520 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 05:03:22.887526 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 05:03:22.887600 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 05:03:22.887699 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0806 05:03:22.888294 (+ 595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:03:22.888953 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:03:22.890045 (+ 1092us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:03:22.890439 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:03:22.890566 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:03:22.893840 (+ 3274us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 05:03:22.893849 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 05:03:22.893853 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 05:03:22.893893 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0806 05:03:22.897059 (+ 3166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 05:03:22.897669 (+ 610us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 05:03:22.897675 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 05:03:22.897678 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 05:03:22.897810 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 05:03:22.898155 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 05:03:22.898160 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 05:03:22.898163 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 05:03:22.898540 (+ 377us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 05:03:22.898687 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0806 05:03:22.899146 (+ 459us) server_negotiation.cc:299] Negotiation successful
0806 05:03:22.899614 (+ 468us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":156,"server-negotiator.queue_time_us":64}
I20260806 05:03:22.901538 11738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:03:22.886015 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55794 (local address 127.9.96.125:34707)
0806 05:03:22.886162 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0806 05:03:22.886168 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 05:03:22.886318 (+ 150us) server_negotiation.cc:407] Connection header received
0806 05:03:22.886480 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 05:03:22.886485 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 05:03:22.886554 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 05:03:22.886627 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0806 05:03:22.888035 (+ 1408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:03:22.888732 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:03:22.889893 (+ 1161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:03:22.890309 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:03:22.890449 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:03:22.895255 (+ 4806us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 05:03:22.895265 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 05:03:22.895272 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 05:03:22.895313 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0806 05:03:22.898508 (+ 3195us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 05:03:22.899361 (+ 853us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 05:03:22.899366 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 05:03:22.899370 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 05:03:22.899565 (+ 195us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 05:03:22.900176 (+ 611us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 05:03:22.900183 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 05:03:22.900186 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 05:03:22.900509 (+ 323us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 05:03:22.900804 (+ 295us) server_negotiation.cc:1035] Waiting for connection context
0806 05:03:22.901038 (+ 234us) server_negotiation.cc:299] Negotiation successful
0806 05:03:22.901233 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260806 05:03:22.905679 11837 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f14c3b06acd44971bbd9af7750df7af4: Bootstrap starting.
I20260806 05:03:22.908236 9601 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 05:03:22.909442 11837 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f14c3b06acd44971bbd9af7750df7af4: Neither blocks nor log segments found. Creating new log.
I20260806 05:03:22.910570 11837 log.cc:824] T 00000000000000000000000000000000 P f14c3b06acd44971bbd9af7750df7af4: Log is configured to *not* fsync() on all Append() calls
I20260806 05:03:22.913837 11837 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f14c3b06acd44971bbd9af7750df7af4: No bootstrap required, opened a new log
I20260806 05:03:22.910668 11578 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "49d7212a4740491a803a558fc032823a"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52430
I20260806 05:03:22.924520 11837 raft_consensus.cc:348] T 00000000000000000000000000000000 P f14c3b06acd44971bbd9af7750df7af4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49d7212a4740491a803a558fc032823a" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 33315 } } peers { permanent_uuid: "5a65793af729464ca850ffef626261fc" member_type: VOTER last_known_addr { host: "127.9.96.125" port: 34707 } } peers { permanent_uuid: "f14c3b06acd44971bbd9af7750df7af4" member_type: VOTER last_known_addr { host: "127.9.96.124" port: 44057 } }
I20260806 05:03:22.924938 11837 raft_consensus.cc:729] T 00000000000000000000000000000000 P f14c3b06acd44971bbd9af7750df7af4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f14c3b06acd44971bbd9af7750df7af4, State: Initialized, Role: FOLLOWER
I20260806 05:03:22.925436 11837 consensus_queue.cc:260] T 00000000000000000000000000000000 P f14c3b06acd44971bbd9af7750df7af4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49d7212a4740491a803a558fc032823a" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 33315 } } peers { permanent_uuid: "5a65793af729464ca850ffef626261fc" member_type: VOTER last_known_addr { host: "127.9.96.125" port: 34707 } } peers { permanent_uuid: "f14c3b06acd44971bbd9af7750df7af4" member_type: VOTER last_known_addr { host: "127.9.96.124" port: 44057 } }
I20260806 05:03:22.926385 11847 sys_catalog.cc:455] T 00000000000000000000000000000000 P f14c3b06acd44971bbd9af7750df7af4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49d7212a4740491a803a558fc032823a" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 33315 } } peers { permanent_uuid: "5a65793af729464ca850ffef626261fc" member_type: VOTER last_known_addr { host: "127.9.96.125" port: 34707 } } peers { permanent_uuid: "f14c3b06acd44971bbd9af7750df7af4" member_type: VOTER last_known_addr { host: "127.9.96.124" port: 44057 } } }
I20260806 05:03:22.926736 11847 sys_catalog.cc:458] T 00000000000000000000000000000000 P f14c3b06acd44971bbd9af7750df7af4 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 05:03:22.927654 11837 sys_catalog.cc:564] T 00000000000000000000000000000000 P f14c3b06acd44971bbd9af7750df7af4 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 05:03:22.928153 11618 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.96.124" port: 44057 } attempt: 4
Aug 06 05:03:22 dist-test-slave-x4qh krb5kdc[11495](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992601, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.96.126@KRBTEST.COM for kudu/127.9.96.124@KRBTEST.COM
I20260806 05:03:22.936110 11742 master_runner.cc:186] Error getting master registration for 127.9.96.126:33315: OK, instance_id { permanent_uuid: "49d7212a4740491a803a558fc032823a" instance_seqno: 1785992601787977 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260806 05:03:22.946610 11742 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260806 05:03:22.947779 11622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:03:22.928924 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.124:44057 (local address 127.0.0.1:47606)
0806 05:03:22.929221 (+ 297us) negotiation.cc:107] Waiting for socket to connect
0806 05:03:22.929245 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0806 05:03:22.929417 (+ 172us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 05:03:22.929946 (+ 529us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 05:03:22.929951 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 05:03:22.930517 (+ 566us) client_negotiation.cc:189] Negotiated authn=SASL
0806 05:03:22.930723 (+ 206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 05:03:22.930735 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 05:03:22.932541 (+ 1806us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 05:03:22.932549 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 05:03:22.933542 (+ 993us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 05:03:22.933555 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 05:03:22.934663 (+ 1108us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 05:03:22.934670 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 05:03:22.934906 (+ 236us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:03:22.935662 (+ 756us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 05:03:22.935691 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0806 05:03:22.939258 (+ 3567us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 05:03:22.943823 (+ 4565us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 05:03:22.943832 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 05:03:22.943840 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0806 05:03:22.944950 (+ 1110us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 05:03:22.945427 (+ 477us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 05:03:22.945435 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 05:03:22.945467 (+ 32us) client_negotiation.cc:760] calling sasl_client_step()
0806 05:03:22.945764 (+ 297us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 05:03:22.946622 (+ 858us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 05:03:22.946630 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 05:03:22.946855 (+ 225us) client_negotiation.cc:769] Sending connection context
0806 05:03:22.947036 (+ 181us) client_negotiation.cc:240] Negotiation successful
0806 05:03:22.947392 (+ 356us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151}
I20260806 05:03:22.948640 11841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:03:22.928996 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47606 (local address 127.9.96.124:44057)
0806 05:03:22.929404 (+ 408us) server_negotiation.cc:206] Beginning negotiation
0806 05:03:22.929412 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 05:03:22.929438 (+ 26us) server_negotiation.cc:407] Connection header received
0806 05:03:22.929551 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 05:03:22.929561 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 05:03:22.929651 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 05:03:22.929756 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0806 05:03:22.930888 (+ 1132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:03:22.932357 (+ 1469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:03:22.933705 (+ 1348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:03:22.934481 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:03:22.934624 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:03:22.939514 (+ 4890us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 05:03:22.939521 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 05:03:22.939526 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 05:03:22.939564 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0806 05:03:22.943611 (+ 4047us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 05:03:22.945108 (+ 1497us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 05:03:22.945114 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 05:03:22.945118 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 05:03:22.945280 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 05:03:22.945943 (+ 663us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 05:03:22.945948 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 05:03:22.945952 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 05:03:22.946371 (+ 419us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 05:03:22.946542 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0806 05:03:22.947225 (+ 683us) server_negotiation.cc:299] Negotiation successful
0806 05:03:22.947629 (+ 404us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":231}
W20260806 05:03:22.949554 11863 catalog_manager.cc:1558] T 00000000000000000000000000000000 P f14c3b06acd44971bbd9af7750df7af4: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 05:03:22.949676 11863 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 05:03:22.954804 11618 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 49d7212a4740491a803a558fc032823a: Bootstrap starting.
I20260806 05:03:22.958724 11618 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 49d7212a4740491a803a558fc032823a: Neither blocks nor log segments found. Creating new log.
I20260806 05:03:22.959980 11618 log.cc:824] T 00000000000000000000000000000000 P 49d7212a4740491a803a558fc032823a: Log is configured to *not* fsync() on all Append() calls
I20260806 05:03:22.963266 11618 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 49d7212a4740491a803a558fc032823a: No bootstrap required, opened a new log
I20260806 05:03:22.970806 11618 raft_consensus.cc:348] T 00000000000000000000000000000000 P 49d7212a4740491a803a558fc032823a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49d7212a4740491a803a558fc032823a" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 33315 } } peers { permanent_uuid: "5a65793af729464ca850ffef626261fc" member_type: VOTER last_known_addr { host: "127.9.96.125" port: 34707 } } peers { permanent_uuid: "f14c3b06acd44971bbd9af7750df7af4" member_type: VOTER last_known_addr { host: "127.9.96.124" port: 44057 } }
I20260806 05:03:22.971195 11618 raft_consensus.cc:729] T 00000000000000000000000000000000 P 49d7212a4740491a803a558fc032823a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49d7212a4740491a803a558fc032823a, State: Initialized, Role: FOLLOWER
I20260806 05:03:22.971724 11618 consensus_queue.cc:260] T 00000000000000000000000000000000 P 49d7212a4740491a803a558fc032823a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49d7212a4740491a803a558fc032823a" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 33315 } } peers { permanent_uuid: "5a65793af729464ca850ffef626261fc" member_type: VOTER last_known_addr { host: "127.9.96.125" port: 34707 } } peers { permanent_uuid: "f14c3b06acd44971bbd9af7750df7af4" member_type: VOTER last_known_addr { host: "127.9.96.124" port: 44057 } }
I20260806 05:03:22.972975 11868 sys_catalog.cc:455] T 00000000000000000000000000000000 P 49d7212a4740491a803a558fc032823a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49d7212a4740491a803a558fc032823a" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 33315 } } peers { permanent_uuid: "5a65793af729464ca850ffef626261fc" member_type: VOTER last_known_addr { host: "127.9.96.125" port: 34707 } } peers { permanent_uuid: "f14c3b06acd44971bbd9af7750df7af4" member_type: VOTER last_known_addr { host: "127.9.96.124" port: 44057 } } }
I20260806 05:03:22.973647 11868 sys_catalog.cc:458] T 00000000000000000000000000000000 P 49d7212a4740491a803a558fc032823a [sys.catalog]: This master's current role is: FOLLOWER
I20260806 05:03:22.974182 11618 sys_catalog.cc:564] T 00000000000000000000000000000000 P 49d7212a4740491a803a558fc032823a [sys.catalog]: configured and running, proceeding with master startup.
I20260806 05:03:22.974248 11578 raft_consensus.cc:482] T 00000000000000000000000000000000 P 49d7212a4740491a803a558fc032823a [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260806 05:03:22.974450 11578 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 49d7212a4740491a803a558fc032823a [term 0 FOLLOWER]: Advancing to term 1
I20260806 05:03:22.978461 11578 raft_consensus.cc:504] T 00000000000000000000000000000000 P 49d7212a4740491a803a558fc032823a [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49d7212a4740491a803a558fc032823a" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 33315 } } peers { permanent_uuid: "5a65793af729464ca850ffef626261fc" member_type: VOTER last_known_addr { host: "127.9.96.125" port: 34707 } } peers { permanent_uuid: "f14c3b06acd44971bbd9af7750df7af4" member_type: VOTER last_known_addr { host: "127.9.96.124" port: 44057 } }
I20260806 05:03:22.981980 11806 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "49d7212a4740491a803a558fc032823a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "f14c3b06acd44971bbd9af7750df7af4"
I20260806 05:03:22.982496 11806 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f14c3b06acd44971bbd9af7750df7af4 [term 0 FOLLOWER]: Advancing to term 1
I20260806 05:03:22.986347 11806 raft_consensus.cc:2454] T 00000000000000000000000000000000 P f14c3b06acd44971bbd9af7750df7af4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49d7212a4740491a803a558fc032823a in term 1.
I20260806 05:03:22.987008 11542 leader_election.cc:302] T 00000000000000000000000000000000 P 49d7212a4740491a803a558fc032823a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49d7212a4740491a803a558fc032823a, f14c3b06acd44971bbd9af7750df7af4; no voters:
I20260806 05:03:22.980700 11578 leader_election.cc:288] T 00000000000000000000000000000000 P 49d7212a4740491a803a558fc032823a [CANDIDATE]: Term 1 election: Requested vote from peers 5a65793af729464ca850ffef626261fc (127.9.96.125:34707), f14c3b06acd44971bbd9af7750df7af4 (127.9.96.124:44057)
I20260806 05:03:22.995420 11868 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 49d7212a4740491a803a558fc032823a [term 1 FOLLOWER]: Leader election won for term 1
I20260806 05:03:22.985062 11514 master_runner.cc:186] Error getting master registration for 127.9.96.125:34707: OK, instance_id { permanent_uuid: "5a65793af729464ca850ffef626261fc" instance_seqno: 1785992602314702 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260806 05:03:22.997884 11514 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260806 05:03:23.002581 11868 raft_consensus.cc:686] T 00000000000000000000000000000000 P 49d7212a4740491a803a558fc032823a [term 1 LEADER]: Becoming Leader. State: Replica: 49d7212a4740491a803a558fc032823a, State: Running, Role: LEADER
I20260806 05:03:23.004359 11868 consensus_queue.cc:237] T 00000000000000000000000000000000 P 49d7212a4740491a803a558fc032823a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49d7212a4740491a803a558fc032823a" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 33315 } } peers { permanent_uuid: "5a65793af729464ca850ffef626261fc" member_type: VOTER last_known_addr { host: "127.9.96.125" port: 34707 } } peers { permanent_uuid: "f14c3b06acd44971bbd9af7750df7af4" member_type: VOTER last_known_addr { host: "127.9.96.124" port: 44057 } }
I20260806 05:03:23.010565 11885 sys_catalog.cc:455] T 00000000000000000000000000000000 P 49d7212a4740491a803a558fc032823a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 49d7212a4740491a803a558fc032823a. Latest consensus state: current_term: 1 leader_uuid: "49d7212a4740491a803a558fc032823a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49d7212a4740491a803a558fc032823a" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 33315 } } peers { permanent_uuid: "5a65793af729464ca850ffef626261fc" member_type: VOTER last_known_addr { host: "127.9.96.125" port: 34707 } } peers { permanent_uuid: "f14c3b06acd44971bbd9af7750df7af4" member_type: VOTER last_known_addr { host: "127.9.96.124" port: 44057 } } }
I20260806 05:03:23.011003 11885 sys_catalog.cc:458] T 00000000000000000000000000000000 P 49d7212a4740491a803a558fc032823a [sys.catalog]: This master's current role is: LEADER
W20260806 05:03:23.016719 11881 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 49d7212a4740491a803a558fc032823a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 05:03:23.016858 11881 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 05:03:23.022847 11894 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 05:03:23.025194 11894 catalog_manager.cc:1484] Initializing Kudu cluster ID...
W20260806 05:03:23.027273 11547 tablet.cc:2369] T 00000000000000000000000000000000 P 49d7212a4740491a803a558fc032823a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 05:03:23.039422 11732 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.96.124" port: 44057 } attempt: 5
I20260806 05:03:23.051606 11738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:03:23.012253 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55800 (local address 127.9.96.125:34707)
0806 05:03:23.012460 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0806 05:03:23.012466 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 05:03:23.024366 (+ 11900us) server_negotiation.cc:407] Connection header received
0806 05:03:23.024755 (+ 389us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 05:03:23.024763 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 05:03:23.024847 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 05:03:23.024924 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0806 05:03:23.031964 (+ 7040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:03:23.033071 (+ 1107us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:03:23.036163 (+ 3092us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:03:23.036839 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:03:23.036954 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:03:23.042197 (+ 5243us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 05:03:23.042204 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 05:03:23.042209 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 05:03:23.042249 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0806 05:03:23.045972 (+ 3723us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 05:03:23.048095 (+ 2123us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 05:03:23.048123 (+ 28us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 05:03:23.048127 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 05:03:23.048386 (+ 259us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 05:03:23.049075 (+ 689us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 05:03:23.049081 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 05:03:23.049084 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 05:03:23.049488 (+ 404us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 05:03:23.049727 (+ 239us) server_negotiation.cc:1035] Waiting for connection context
0806 05:03:23.050638 (+ 911us) server_negotiation.cc:299] Negotiation successful
0806 05:03:23.050988 (+ 350us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260806 05:03:23.051903 11844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:03:23.012449 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52446 (local address 127.9.96.126:33315)
0806 05:03:23.012667 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0806 05:03:23.012675 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 05:03:23.023976 (+ 11301us) server_negotiation.cc:407] Connection header received
0806 05:03:23.024254 (+ 278us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 05:03:23.024260 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 05:03:23.024340 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 05:03:23.024434 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0806 05:03:23.029436 (+ 5002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:03:23.030535 (+ 1099us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:03:23.033059 (+ 2524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:03:23.033792 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:03:23.033888 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:03:23.042361 (+ 8473us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 05:03:23.042367 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 05:03:23.042372 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 05:03:23.042414 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0806 05:03:23.047749 (+ 5335us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 05:03:23.048832 (+ 1083us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 05:03:23.048839 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 05:03:23.048843 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 05:03:23.048995 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 05:03:23.049500 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 05:03:23.049508 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 05:03:23.049514 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 05:03:23.049976 (+ 462us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 05:03:23.050211 (+ 235us) server_negotiation.cc:1035] Waiting for connection context
0806 05:03:23.051274 (+ 1063us) server_negotiation.cc:299] Negotiation successful
0806 05:03:23.051534 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260806 05:03:23.053867 11841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:03:23.012265 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47622 (local address 127.9.96.124:44057)
0806 05:03:23.012628 (+ 363us) server_negotiation.cc:206] Beginning negotiation
0806 05:03:23.012636 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 05:03:23.018724 (+ 6088us) server_negotiation.cc:407] Connection header received
0806 05:03:23.019749 (+ 1025us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 05:03:23.019755 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 05:03:23.019886 (+ 131us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 05:03:23.019964 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0806 05:03:23.027690 (+ 7726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:03:23.028808 (+ 1118us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:03:23.031361 (+ 2553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:03:23.032043 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:03:23.032186 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:03:23.037371 (+ 5185us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 05:03:23.037383 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 05:03:23.037388 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 05:03:23.037436 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0806 05:03:23.042778 (+ 5342us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 05:03:23.049979 (+ 7201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 05:03:23.049985 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 05:03:23.049989 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 05:03:23.050148 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 05:03:23.051336 (+ 1188us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 05:03:23.051341 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 05:03:23.051344 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 05:03:23.051708 (+ 364us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 05:03:23.051955 (+ 247us) server_negotiation.cc:1035] Waiting for connection context
0806 05:03:23.053263 (+ 1308us) server_negotiation.cc:299] Negotiation successful
0806 05:03:23.053498 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":243}
I20260806 05:03:23.068812 11806 raft_consensus.cc:1261] T 00000000000000000000000000000000 P f14c3b06acd44971bbd9af7750df7af4 [term 1 FOLLOWER]: Refusing update from remote peer 49d7212a4740491a803a558fc032823a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260806 05:03:23.068897 11775 tablet.cc:2369] T 00000000000000000000000000000000 P f14c3b06acd44971bbd9af7750df7af4: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 06 05:03:23 dist-test-slave-x4qh krb5kdc[11495](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785992602, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.96.125@KRBTEST.COM for kudu/127.9.96.124@KRBTEST.COM
I20260806 05:03:23.070420 11885 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 49d7212a4740491a803a558fc032823a [LEADER]: Connected to new peer: Peer: permanent_uuid: "f14c3b06acd44971bbd9af7750df7af4" member_type: VOTER last_known_addr { host: "127.9.96.124" port: 44057 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 05:03:23.084617 11735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:03:23.040001 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.96.124:44057 (local address 127.0.0.1:47636)
0806 05:03:23.040397 (+ 396us) negotiation.cc:107] Waiting for socket to connect
0806 05:03:23.040418 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0806 05:03:23.040526 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 05:03:23.054504 (+ 13978us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 05:03:23.054509 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 05:03:23.054990 (+ 481us) client_negotiation.cc:189] Negotiated authn=SASL
0806 05:03:23.055193 (+ 203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 05:03:23.055205 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 05:03:23.056755 (+ 1550us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 05:03:23.056761 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 05:03:23.058203 (+ 1442us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 05:03:23.058219 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 05:03:23.064215 (+ 5996us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 05:03:23.064222 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 05:03:23.064497 (+ 275us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:03:23.065108 (+ 611us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 05:03:23.065132 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0806 05:03:23.072558 (+ 7426us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 05:03:23.081160 (+ 8602us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 05:03:23.081169 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 05:03:23.081176 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0806 05:03:23.081870 (+ 694us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 05:03:23.082478 (+ 608us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 05:03:23.082486 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 05:03:23.082492 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0806 05:03:23.082727 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 05:03:23.083478 (+ 751us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 05:03:23.083484 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 05:03:23.083650 (+ 166us) client_negotiation.cc:769] Sending connection context
0806 05:03:23.083818 (+ 168us) client_negotiation.cc:240] Negotiation successful
0806 05:03:23.084108 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":268}
I20260806 05:03:23.085814 11841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:03:23.040838 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47636 (local address 127.9.96.124:44057)
0806 05:03:23.054176 (+ 13338us) server_negotiation.cc:206] Beginning negotiation
0806 05:03:23.054182 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 05:03:23.054199 (+ 17us) server_negotiation.cc:407] Connection header received
0806 05:03:23.054247 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 05:03:23.054257 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 05:03:23.054326 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 05:03:23.055341 (+ 1015us) server_negotiation.cc:226] Negotiated authn=SASL
0806 05:03:23.055553 (+ 212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:03:23.056606 (+ 1053us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:03:23.060348 (+ 3742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:03:23.063955 (+ 3607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:03:23.064842 (+ 887us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:03:23.073025 (+ 8183us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 05:03:23.073035 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 05:03:23.073040 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 05:03:23.073106 (+ 66us) server_negotiation.cc:922] calling sasl_server_start()
0806 05:03:23.080961 (+ 7855us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 05:03:23.082078 (+ 1117us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 05:03:23.082087 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 05:03:23.082093 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 05:03:23.082319 (+ 226us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 05:03:23.082892 (+ 573us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 05:03:23.082897 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 05:03:23.082900 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 05:03:23.083307 (+ 407us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 05:03:23.084997 (+ 1690us) server_negotiation.cc:1035] Waiting for connection context
0806 05:03:23.085129 (+ 132us) server_negotiation.cc:299] Negotiation successful
0806 05:03:23.085378 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13221,"thread_start_us":22229,"threads_started":1}
I20260806 05:03:23.093088 11732 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5a65793af729464ca850ffef626261fc: Bootstrap starting.
I20260806 05:03:23.098201 11732 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5a65793af729464ca850ffef626261fc: Neither blocks nor log segments found. Creating new log.
I20260806 05:03:23.100193 11732 log.cc:824] T 00000000000000000000000000000000 P 5a65793af729464ca850ffef626261fc: Log is configured to *not* fsync() on all Append() calls
I20260806 05:03:23.103937 11847 sys_catalog.cc:455] T 00000000000000000000000000000000 P f14c3b06acd44971bbd9af7750df7af4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 49d7212a4740491a803a558fc032823a. Latest consensus state: current_term: 1 leader_uuid: "49d7212a4740491a803a558fc032823a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49d7212a4740491a803a558fc032823a" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 33315 } } peers { permanent_uuid: "5a65793af729464ca850ffef626261fc" member_type: VOTER last_known_addr { host: "127.9.96.125" port: 34707 } } peers { permanent_uuid: "f14c3b06acd44971bbd9af7750df7af4" member_type: VOTER last_known_addr { host: "127.9.96.124" port: 44057 } } }
I20260806 05:03:23.104290 11847 sys_catalog.cc:458] T 00000000000000000000000000000000 P f14c3b06acd44971bbd9af7750df7af4 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 05:03:23.107841 11900 mvcc.cc:205] Tried to move back new op lower bound from 7315425702157246464 to 7315425701911330816. Current Snapshot: MvccSnapshot[applied={T|T < 7315425702157246464}]
I20260806 05:03:23.109311 11847 sys_catalog.cc:455] T 00000000000000000000000000000000 P f14c3b06acd44971bbd9af7750df7af4 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "49d7212a4740491a803a558fc032823a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49d7212a4740491a803a558fc032823a" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 33315 } } peers { permanent_uuid: "5a65793af729464ca850ffef626261fc" member_type: VOTER last_known_addr { host: "127.9.96.125" port: 34707 } } peers { permanent_uuid: "f14c3b06acd44971bbd9af7750df7af4" member_type: VOTER last_known_addr { host: "127.9.96.124" port: 44057 } } }
I20260806 05:03:23.109550 11847 sys_catalog.cc:458] T 00000000000000000000000000000000 P f14c3b06acd44971bbd9af7750df7af4 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 05:03:23.110977 11732 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5a65793af729464ca850ffef626261fc: No bootstrap required, opened a new log
I20260806 05:03:23.123600 11732 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5a65793af729464ca850ffef626261fc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49d7212a4740491a803a558fc032823a" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 33315 } } peers { permanent_uuid: "5a65793af729464ca850ffef626261fc" member_type: VOTER last_known_addr { host: "127.9.96.125" port: 34707 } } peers { permanent_uuid: "f14c3b06acd44971bbd9af7750df7af4" member_type: VOTER last_known_addr { host: "127.9.96.124" port: 44057 } }
I20260806 05:03:23.123991 11732 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5a65793af729464ca850ffef626261fc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a65793af729464ca850ffef626261fc, State: Initialized, Role: FOLLOWER
I20260806 05:03:23.124508 11732 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5a65793af729464ca850ffef626261fc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49d7212a4740491a803a558fc032823a" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 33315 } } peers { permanent_uuid: "5a65793af729464ca850ffef626261fc" member_type: VOTER last_known_addr { host: "127.9.96.125" port: 34707 } } peers { permanent_uuid: "f14c3b06acd44971bbd9af7750df7af4" member_type: VOTER last_known_addr { host: "127.9.96.124" port: 44057 } }
I20260806 05:03:23.125296 11885 sys_catalog.cc:455] T 00000000000000000000000000000000 P 49d7212a4740491a803a558fc032823a [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "49d7212a4740491a803a558fc032823a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49d7212a4740491a803a558fc032823a" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 33315 } } peers { permanent_uuid: "5a65793af729464ca850ffef626261fc" member_type: VOTER last_known_addr { host: "127.9.96.125" port: 34707 } } peers { permanent_uuid: "f14c3b06acd44971bbd9af7750df7af4" member_type: VOTER last_known_addr { host: "127.9.96.124" port: 44057 } } }
I20260806 05:03:23.125671 11885 sys_catalog.cc:458] T 00000000000000000000000000000000 P 49d7212a4740491a803a558fc032823a [sys.catalog]: This master's current role is: LEADER
I20260806 05:03:23.127617 11894 catalog_manager.cc:1347] Generated new cluster ID: 644fe6df7a71450e9e16e8d0c37ba1b4
I20260806 05:03:23.127744 11894 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 05:03:23.162973 11732 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5a65793af729464ca850ffef626261fc [sys.catalog]: configured and running, proceeding with master startup.
I20260806 05:03:23.164749 11688 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "49d7212a4740491a803a558fc032823a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "5a65793af729464ca850ffef626261fc"
I20260806 05:03:23.168891 11688 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5a65793af729464ca850ffef626261fc [term 0 FOLLOWER]: Advancing to term 1
I20260806 05:03:23.173552 11688 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 5a65793af729464ca850ffef626261fc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49d7212a4740491a803a558fc032823a in term 1.
I20260806 05:03:23.176052 11687 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 5a65793af729464ca850ffef626261fc [term 1 FOLLOWER]: Refusing update from remote peer 49d7212a4740491a803a558fc032823a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 05:03:23.177871 11909 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 49d7212a4740491a803a558fc032823a [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a65793af729464ca850ffef626261fc" member_type: VOTER last_known_addr { host: "127.9.96.125" port: 34707 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 05:03:23.187156 11907 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5a65793af729464ca850ffef626261fc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49d7212a4740491a803a558fc032823a" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 33315 } } peers { permanent_uuid: "5a65793af729464ca850ffef626261fc" member_type: VOTER last_known_addr { host: "127.9.96.125" port: 34707 } } peers { permanent_uuid: "f14c3b06acd44971bbd9af7750df7af4" member_type: VOTER last_known_addr { host: "127.9.96.124" port: 44057 } } }
I20260806 05:03:23.189002 11907 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5a65793af729464ca850ffef626261fc [sys.catalog]: This master's current role is: FOLLOWER
I20260806 05:03:23.220014 11894 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 05:03:23.222506 11894 catalog_manager.cc:1504] Loading token signing keys...
I20260806 05:03:23.239434 11907 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5a65793af729464ca850ffef626261fc [sys.catalog]: SysCatalogTable state changed. Reason: New leader 49d7212a4740491a803a558fc032823a. Latest consensus state: current_term: 1 leader_uuid: "49d7212a4740491a803a558fc032823a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49d7212a4740491a803a558fc032823a" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 33315 } } peers { permanent_uuid: "5a65793af729464ca850ffef626261fc" member_type: VOTER last_known_addr { host: "127.9.96.125" port: 34707 } } peers { permanent_uuid: "f14c3b06acd44971bbd9af7750df7af4" member_type: VOTER last_known_addr { host: "127.9.96.124" port: 44057 } } }
I20260806 05:03:23.239980 11907 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5a65793af729464ca850ffef626261fc [sys.catalog]: This master's current role is: FOLLOWER
I20260806 05:03:23.240236 11917 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5a65793af729464ca850ffef626261fc [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "49d7212a4740491a803a558fc032823a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49d7212a4740491a803a558fc032823a" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 33315 } } peers { permanent_uuid: "5a65793af729464ca850ffef626261fc" member_type: VOTER last_known_addr { host: "127.9.96.125" port: 34707 } } peers { permanent_uuid: "f14c3b06acd44971bbd9af7750df7af4" member_type: VOTER last_known_addr { host: "127.9.96.124" port: 44057 } } }
I20260806 05:03:23.240633 11917 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5a65793af729464ca850ffef626261fc [sys.catalog]: This master's current role is: FOLLOWER
I20260806 05:03:23.241905 11909 sys_catalog.cc:455] T 00000000000000000000000000000000 P 49d7212a4740491a803a558fc032823a [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "49d7212a4740491a803a558fc032823a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49d7212a4740491a803a558fc032823a" member_type: VOTER last_known_addr { host: "127.9.96.126" port: 33315 } } peers { permanent_uuid: "5a65793af729464ca850ffef626261fc" member_type: VOTER last_known_addr { host: "127.9.96.125" port: 34707 } } peers { permanent_uuid: "f14c3b06acd44971bbd9af7750df7af4" member_type: VOTER last_known_addr { host: "127.9.96.124" port: 44057 } } }
I20260806 05:03:23.242307 11909 sys_catalog.cc:458] T 00000000000000000000000000000000 P 49d7212a4740491a803a558fc032823a [sys.catalog]: This master's current role is: LEADER
I20260806 05:03:23.259449 11922 mvcc.cc:205] Tried to move back new op lower bound from 7315425702157246464 to 7315425701911330816. Current Snapshot: MvccSnapshot[applied={T|T < 7315425702157246464}]
I20260806 05:03:23.266075 11934 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260806 05:03:23.270839 11894 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 49d7212a4740491a803a558fc032823a: Generated new TSK 0
I20260806 05:03:23.271817 11894 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260806 05:03:23.382035 11841 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 05:03:23.371162 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47648 (local address 127.9.96.124:44057)
0806 05:03:23.371470 (+ 308us) server_negotiation.cc:206] Beginning negotiation
0806 05:03:23.371479 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 05:03:23.379953 (+ 8474us) server_negotiation.cc:407] Connection header received
0806 05:03:23.380260 (+ 307us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 05:03:23.380266 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 05:03:23.380385 (+ 119us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 05:03:23.380486 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0806 05:03:23.381784 (+ 1298us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47648: BlockingRecv error: recv got EOF from 127.0.0.1:47648 (error 108)
Metrics: {"server-negotiator.queue_time_us":82}
W20260806 05:03:23.383191 11738 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 05:03:23.371960 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55804 (local address 127.9.96.125:34707)
0806 05:03:23.372158 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0806 05:03:23.372164 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 05:03:23.380796 (+ 8632us) server_negotiation.cc:407] Connection header received
0806 05:03:23.381122 (+ 326us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 05:03:23.381132 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 05:03:23.381273 (+ 141us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 05:03:23.381362 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0806 05:03:23.382898 (+ 1536us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55804: BlockingRecv error: recv got EOF from 127.0.0.1:55804 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
W20260806 05:03:23.384684 11844 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 05:03:23.370818 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52452 (local address 127.9.96.126:33315)
0806 05:03:23.371003 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0806 05:03:23.371010 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 05:03:23.382997 (+ 11987us) server_negotiation.cc:407] Connection header received
0806 05:03:23.383154 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 05:03:23.383160 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 05:03:23.383265 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 05:03:23.383394 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0806 05:03:23.384362 (+ 968us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52452: BlockingRecv error: recv got EOF from 127.0.0.1:52452 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
W20260806 05:03:23.400113 11738 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 05:03:23.397489 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55810 (local address 127.9.96.125:34707)
0806 05:03:23.397669 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0806 05:03:23.397675 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 05:03:23.398173 (+ 498us) server_negotiation.cc:407] Connection header received
0806 05:03:23.398500 (+ 327us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 05:03:23.398508 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 05:03:23.398644 (+ 136us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 05:03:23.398728 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0806 05:03:23.399802 (+ 1074us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55810: BlockingRecv error: recv got EOF from 127.0.0.1:55810 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
W20260806 05:03:23.415135 11738 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 05:03:23.412957 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55822 (local address 127.9.96.125:34707)
0806 05:03:23.413125 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0806 05:03:23.413131 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 05:03:23.413647 (+ 516us) server_negotiation.cc:407] Connection header received
0806 05:03:23.413823 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 05:03:23.413830 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 05:03:23.413933 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 05:03:23.414054 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0806 05:03:23.414926 (+ 872us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55822: BlockingRecv error: recv got EOF from 127.0.0.1:55822 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
W20260806 05:03:23.431077 11738 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 05:03:23.428926 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55832 (local address 127.9.96.125:34707)
0806 05:03:23.429093 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0806 05:03:23.429099 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 05:03:23.429602 (+ 503us) server_negotiation.cc:407] Connection header received
0806 05:03:23.429783 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 05:03:23.429790 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 05:03:23.429897 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 05:03:23.430018 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0806 05:03:23.430831 (+ 813us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55832: BlockingRecv error: recv got EOF from 127.0.0.1:55832 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
W20260806 05:03:23.440303 11738 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 05:03:23.438111 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55846 (local address 127.9.96.125:34707)
0806 05:03:23.438268 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0806 05:03:23.438275 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 05:03:23.438805 (+ 530us) server_negotiation.cc:407] Connection header received
0806 05:03:23.438993 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 05:03:23.438999 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 05:03:23.439095 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 05:03:23.439203 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0806 05:03:23.440089 (+ 886us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55846: BlockingRecv error: recv got EOF from 127.0.0.1:55846 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
W20260806 05:03:23.455303 11738 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 05:03:23.453077 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55852 (local address 127.9.96.125:34707)
0806 05:03:23.453256 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0806 05:03:23.453262 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 05:03:23.453793 (+ 531us) server_negotiation.cc:407] Connection header received
0806 05:03:23.454010 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 05:03:23.454016 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 05:03:23.454110 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 05:03:23.454232 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0806 05:03:23.455067 (+ 835us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55852: BlockingRecv error: recv got EOF from 127.0.0.1:55852 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
W20260806 05:03:23.478839 11738 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 05:03:23.476410 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55860 (local address 127.9.96.125:34707)
0806 05:03:23.476618 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0806 05:03:23.476624 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 05:03:23.477194 (+ 570us) server_negotiation.cc:407] Connection header received
0806 05:03:23.477428 (+ 234us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 05:03:23.477435 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 05:03:23.477532 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 05:03:23.477650 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0806 05:03:23.478575 (+ 925us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55860: BlockingRecv error: recv got EOF from 127.0.0.1:55860 (error 108)
Metrics: {"server-negotiator.queue_time_us":79}
W20260806 05:03:23.523919 11738 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 05:03:23.521601 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55868 (local address 127.9.96.125:34707)
0806 05:03:23.521800 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0806 05:03:23.521807 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 05:03:23.522370 (+ 563us) server_negotiation.cc:407] Connection header received
0806 05:03:23.522577 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 05:03:23.522584 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 05:03:23.522689 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 05:03:23.522801 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0806 05:03:23.523653 (+ 852us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55868: BlockingRecv error: recv got EOF from 127.0.0.1:55868 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
W20260806 05:03:23.592969 11738 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 05:03:23.591030 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55882 (local address 127.9.96.125:34707)
0806 05:03:23.591212 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0806 05:03:23.591217 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 05:03:23.591652 (+ 435us) server_negotiation.cc:407] Connection header received
0806 05:03:23.591827 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 05:03:23.591831 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 05:03:23.591916 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 05:03:23.592013 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0806 05:03:23.592715 (+ 702us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55882: BlockingRecv error: recv got EOF from 127.0.0.1:55882 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
W20260806 05:03:23.728780 11738 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 05:03:23.726740 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55898 (local address 127.9.96.125:34707)
0806 05:03:23.727018 (+ 278us) server_negotiation.cc:206] Beginning negotiation
0806 05:03:23.727025 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 05:03:23.727292 (+ 267us) server_negotiation.cc:407] Connection header received
0806 05:03:23.727536 (+ 244us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 05:03:23.727540 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 05:03:23.727615 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 05:03:23.727737 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0806 05:03:23.728480 (+ 743us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55898: BlockingRecv error: recv got EOF from 127.0.0.1:55898 (error 108)
Metrics: {"server-negotiator.queue_time_us":103}
I20260806 05:03:23.951737 11863 catalog_manager.cc:1259] Loaded cluster ID: 644fe6df7a71450e9e16e8d0c37ba1b4
I20260806 05:03:23.951987 11863 catalog_manager.cc:1552] T 00000000000000000000000000000000 P f14c3b06acd44971bbd9af7750df7af4: loading cluster ID for follower catalog manager: success
I20260806 05:03:23.954458 11863 catalog_manager.cc:1574] T 00000000000000000000000000000000 P f14c3b06acd44971bbd9af7750df7af4: acquiring CA information for follower catalog manager: success
I20260806 05:03:23.956244 11863 catalog_manager.cc:1602] T 00000000000000000000000000000000 P f14c3b06acd44971bbd9af7750df7af4: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260806 05:03:23.990314 11738 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 05:03:23.988569 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55900 (local address 127.9.96.125:34707)
0806 05:03:23.988758 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0806 05:03:23.988763 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 05:03:23.989086 (+ 323us) server_negotiation.cc:407] Connection header received
0806 05:03:23.989273 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 05:03:23.989278 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 05:03:23.989364 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 05:03:23.989469 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0806 05:03:23.990142 (+ 673us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55900: BlockingRecv error: recv got EOF from 127.0.0.1:55900 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
I20260806 05:03:24.270421 11934 catalog_manager.cc:1259] Loaded cluster ID: 644fe6df7a71450e9e16e8d0c37ba1b4
I20260806 05:03:24.270530 11934 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 5a65793af729464ca850ffef626261fc: loading cluster ID for follower catalog manager: success
I20260806 05:03:24.273403 11934 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 5a65793af729464ca850ffef626261fc: acquiring CA information for follower catalog manager: success
I20260806 05:03:24.275496 11934 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 5a65793af729464ca850ffef626261fc: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 05:03:24.513556 12031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:03:24.506822 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55916 (local address 127.9.96.125:34707)
0806 05:03:24.507699 (+ 877us) server_negotiation.cc:206] Beginning negotiation
0806 05:03:24.507704 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 05:03:24.507719 (+ 15us) server_negotiation.cc:407] Connection header received
0806 05:03:24.507783 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 05:03:24.507787 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 05:03:24.507935 (+ 148us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 05:03:24.508074 (+ 139us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 05:03:24.508576 (+ 502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:03:24.509481 (+ 905us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:03:24.510835 (+ 1354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:03:24.511296 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:03:24.511496 (+ 200us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:03:24.512263 (+ 767us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 05:03:24.512853 (+ 590us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 05:03:24.513039 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0806 05:03:24.513135 (+ 96us) server_negotiation.cc:299] Negotiation successful
0806 05:03:24.513229 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":738,"thread_start_us":566,"threads_started":1}
I20260806 05:03:24.528972 12041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:03:24.519904 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47662 (local address 127.9.96.124:44057)
0806 05:03:24.523726 (+ 3822us) server_negotiation.cc:206] Beginning negotiation
0806 05:03:24.523730 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 05:03:24.523747 (+ 17us) server_negotiation.cc:407] Connection header received
0806 05:03:24.523836 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 05:03:24.523840 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 05:03:24.523983 (+ 143us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 05:03:24.524159 (+ 176us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 05:03:24.524602 (+ 443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:03:24.525303 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:03:24.526663 (+ 1360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:03:24.527073 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:03:24.527211 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:03:24.527803 (+ 592us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 05:03:24.528378 (+ 575us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 05:03:24.528534 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0806 05:03:24.528594 (+ 60us) server_negotiation.cc:299] Negotiation successful
0806 05:03:24.528672 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3715,"thread_start_us":3632,"threads_started":1}
I20260806 05:03:24.554294 12041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:03:24.535912 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47676 (local address 127.9.96.124:44057)
0806 05:03:24.536078 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0806 05:03:24.536087 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 05:03:24.545920 (+ 9833us) server_negotiation.cc:407] Connection header received
0806 05:03:24.546234 (+ 314us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 05:03:24.546243 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 05:03:24.546369 (+ 126us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 05:03:24.546445 (+ 76us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 05:03:24.547356 (+ 911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:03:24.548826 (+ 1470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:03:24.552001 (+ 3175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:03:24.552635 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:03:24.552739 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:03:24.553511 (+ 772us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 05:03:24.553636 (+ 125us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 05:03:24.553696 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0806 05:03:24.553929 (+ 233us) server_negotiation.cc:299] Negotiation successful
0806 05:03:24.554009 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260806 05:03:24.558686 12031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:03:24.535083 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55928 (local address 127.9.96.125:34707)
0806 05:03:24.535270 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0806 05:03:24.535275 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 05:03:24.552052 (+ 16777us) server_negotiation.cc:407] Connection header received
0806 05:03:24.552341 (+ 289us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 05:03:24.552349 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 05:03:24.552444 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 05:03:24.552542 (+ 98us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 05:03:24.553386 (+ 844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:03:24.554457 (+ 1071us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:03:24.556857 (+ 2400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:03:24.557379 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:03:24.557538 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:03:24.557944 (+ 406us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 05:03:24.558047 (+ 103us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 05:03:24.558158 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0806 05:03:24.558328 (+ 170us) server_negotiation.cc:299] Negotiation successful
0806 05:03:24.558404 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260806 05:03:24.560551 12051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 05:03:24.535289 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52456 (local address 127.9.96.126:33315)
0806 05:03:24.552230 (+ 16941us) server_negotiation.cc:206] Beginning negotiation
0806 05:03:24.552238 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 05:03:24.552258 (+ 20us) server_negotiation.cc:407] Connection header received
0806 05:03:24.552381 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 05:03:24.552388 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 05:03:24.552614 (+ 226us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 05:03:24.553582 (+ 968us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 05:03:24.553772 (+ 190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:03:24.554831 (+ 1059us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:03:24.557126 (+ 2295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 05:03:24.557766 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 05:03:24.558409 (+ 643us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 05:03:24.558737 (+ 328us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 05:03:24.559566 (+ 829us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 05:03:24.560119 (+ 553us) server_negotiation.cc:1035] Waiting for connection context
0806 05:03:24.560173 (+ 54us) server_negotiation.cc:299] Negotiation successful
0806 05:03:24.560254 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16789,"thread_start_us":16666,"threads_started":1}
I20260806 05:03:24.565693 9601 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu pid 11514
I20260806 05:03:24.566829 11609 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 05:03:24.680788 9601 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu with pid 11514
I20260806 05:03:24.716482 9601 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu pid 11628
I20260806 05:03:24.717727 11723 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 05:03:24.823408 9601 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu with pid 11628
I20260806 05:03:24.858560 9601 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu pid 11742
I20260806 05:03:24.859634 11832 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 05:03:24.965981 9601 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOfb9uZ/build/asan/bin/kudu with pid 11742
2026-08-06T05:03:24Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (5276 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (10311 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (377841 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST